I have the function $$f(x, y) = \ln(1 + xy) - \frac{1}{3}(x^2+y^2)$$ I found its critical points, one saddle and two maxima, the latter of which are $$A = (1/\sqrt{2}, 1/\sqrt{2}) \qquad \qquad B = -A$$
How can I say that those points are global maxima and not just local, without plotting the function? I thought about analysing what happens when $(x, y) \to (\pm \infty,\ \pm\infty)$ but there are times in which it's a mess.
In this case for example, I thought about studying what happens along the straight lines $x = y$, getting
$$f(x) = \ln(1+x^2) - \frac{2}{3}x^2 \sim 2\ln|x| - x^2$$
as $x \to \pm \infty$, and this either cases goes to $-\infty$.
Hence there is an upper limit and my points are global max. Is this reasoning correct? Or is it wrong and why?
Are there better / "righter" ways to proceed in general, when dealing with functions in two variables in the case of a non compact domain?
Showing that $\lim_{x\to \pm\infty}f(x,x)=-\infty$, you have just proved that the function has no global minimum.
In order to investigate $f$ at infinity, you may also use polar coordinates: letting $x=r\cos(\theta)$, $y=r\sin(\theta)$, we get $$f(x,y)=\ln(1 + r^2\cos(\theta)\sin(\theta)) - \frac{r^2}{3}= \ln\left(1 + \frac{r^2\sin(2\theta)}{2}\right) - \frac{r^2}{3}\leq \ln\left(1 + \frac{r^2}{2}\right) - \frac{r^2}{3}$$ and as $r$ goes to $+\infty$ the RHS goes to $-\infty$ $$\ln\left(1 + \frac{r^2}{2}\right) - \frac{r^2}{3}\sim 2\ln(r) - \frac{r^2}{3}\sim - \frac{r^2}{3}\to -\infty.$$ Therefore $$\lim_{\|(x,y)\|\to +\infty}f(x,y)=-\infty.$$ Here you should also check the limit at any point of the boundary of the (open) domain $D=\{(x,y): xy>-1\}$: if $(x_0,y_0)\in \partial D=\{(x,y): xy=-1\}$, $$\lim_{(x,y)\to (x_0,y_0)}f(x,y)=\ln(0^+) - \frac{1}{3}(x_0^2+y_0^2)=-\infty.$$ Now we have established that the function has global maximum at the critical points $(1/\sqrt{2}, 1/\sqrt{2}) $ and $(-1/\sqrt{2},-1/\sqrt{2}) $.