I'm trying to find maxima/minima points for the following function: $g(x,y) = -5x^2-3y^2+x-xy^2+2$
I found three points, $p_1 =(0.1,0), p_2 =(-3,-\sqrt{31}), p_3= (-3,\sqrt{31})$.
But for two of them the Hessian matrix was equal to zero. I tried to tackle this by trying to find a specific path where $p_2,p_3$ are not saddle points. I haven't been able to find such paths.
I'd appreciate if anyone can tell me if my way is correct, or ideas on how to tackle this problem, or any sort of guidance.