The given functions are $$f(x,y,z)=xy-z$$ $$g(x,y,z)=\frac{3}{4}x^2-y^2-z$$ And the point is $(2,-3,-6)$ So first I found the gradient at the given point for both and got $$f=\left<2,-3,-1\right>$$ $$g=\left<3,6,-1\right>$$ Then their respective magnitudes $$f=\sqrt{14}$$ $$g=\sqrt{46}$$
Now finding the angle of inclination between the two, I first took the dot product of the two gradient vectors and then multiplied the two magnitudes together. Putting the dot product over the product of the magnitudes I got $$\frac{-11}{2\sqrt{161}}$$ Now this is where I get confused because this show be outside the range of the arc-cosine function but when I used an online calculator I got $115.69$ degrees. So is this correct or have I gotten it right?

It is not outside the range as the value is ($ \gt -1)$ but I think you have made a mistake in calculation of gradient of function $f$.
$f(x,y,z)=xy-z$
$\frac{\partial f}{\partial x} = y, \frac{\partial f}{\partial y} = x, \frac{\partial f}{\partial z} = -1$
So gradient of $f = (-3, 2, -1)$
Similarly gradient of $g = (3, 6, -1)$
Dot product $= 4$
So $\cos \theta = \frac{2}{\sqrt 161}$