I'm dealing with an optimization problem, so I need to determine if I get maximum, minimum, or saddle points in a function of two variables. However, When evaluating the eigenvalues o my Hessian Matrix I get one infinite Eigenvalue. What does it mean? Is there any other way to classify the inflection point?
My Hessian matrix actually reads
$$ \mathbf{H} = \left(\begin{array}{cc} x & 0 \\ 0 & 1/y \end{array} \right) \,\, , $$
with $y = 0$ and $x = x^{\star}$ in my fixed point. How could I proceed with this analysis?
Edit: The example in the question is just a toy model. My real Hessian is the following:
$$ \mathbf{H} = \left( \begin{array}{cc} 2\tilde{g}'(r_\star)h(r_\star) & 0 \\ 0 & -2h(r_\star) \end{array} \right) \,\, , $$
Where the function $h(r_\star) \to \infty$ in the critical point. The functions $$ h(r) = \frac{r^3 (3 M-2 r)}{(2 M-r) ((C-4) M+2 r)}\,\, , $$ $$ \tilde{g}'(r) = -\frac{(3 M-r) ((C-4) M+2 r)}{r (3 M-2 r) (2 M-r)} \,\, , $$
and my $r_\star = \frac{1}{2} (4 M-C M)$, Where $C \in \mathbb{R}$ and $M \in \mathbb{R}^+$.
Your Hessian matrix gives us the following:
$$\frac{d^2f(x,y)}{dx^2}=x$$
$$\frac{d^2f(x,y)}{dxdy}=\frac{d^2f(x,y)}{dydx}=0$$
$$\frac{d^2f(x,y)}{dy^2}=1/y$$
From these, we can infer that your function is something like:
$$f(x,y)=\frac{1}{6}x^3+C_1x+y\log y-y+C_2y+C_3$$
This function is not differentiable on the line $y=0$ (since it's not defined for points with $y<0$), so classifying critical points using the Hessian isn't going to work.
That said, you can still figure out whether a point is a maximum or a minimum, just not in the usual algorithmic way. Taking the limit as $y$ approaches zero from the positive direction, we get:
$$\lim_{y\to 0^+}f(x,y)\equiv g(x)=\frac{1}{6}x^3+C_1x+C_3$$
since $\lim_{y\to 0}y\log y=0$. So, if we treat this limit as a function of $x$, we can find where the critical points are on this line by setting $g'(x)=0$:
$$\frac{1}{2}x^2+C_1=0$$
from which we get:
$$x^*=\pm\sqrt{-2C_1}$$
So now we have the critical points in $x$ (if they exist - for $C_1>0$ there are no critical points on this line!). The second derivative is $g''(x)=x$, as the Hessian implied. So $\sqrt{-2C_1}$ is a minimum along the $x$ direction and $-\sqrt{-2C_1}$ is a maximum along the $x$ direction.
Now that we have the behavior along the $x$ direction, we can see what the behavior along the $y$ direction is. So let's take the partial derivative of the original function with respect to $y$:
$$\frac{\partial f(x,y)}{\partial y}=\log y+C_2$$
As you can see, the function clearly decreases with increasing $y$ very close to zero, no matter what the value of $C_2$ is (since $\log y$ decreases to negative infinity as you approach zero). So, no matter what the value of $C_2$ is, the function decreases along the $y$ direction very close to zero.
So, for $(\sqrt{-2C_1},0)$, there is a minimum in $x$ and a maximum in $y$, meaning that this is (sort of) a (one-sided) saddle point. And for $(-\sqrt{-2C_1},0)$, there is a maximum in $x$ and a maximum in $y$, meaning that this is (sort of) a (one-sided) maximum.