I was studying this function: $$f(x)=x^8-2x^4y+y^3-y$$ and I had to compute maxima/minima/saddles. I found the minima by studying gradient and hessian of the function which are: $$\nabla f(x,y)=(8x^7-8x^3y,3y^2-2x^4-1)$$ $$H_f(x,y)=\begin{bmatrix}56x^6-24x^2y & -8x^3 \\ -8x^3 & 6y \end{bmatrix}$$
from which we have that stationary points are: $(0, \pm \frac{1}{\sqrt 3}), (\pm 1,1)$. We have that, as the hessian is positive defined in $(\pm 1, 1)$, that these points are local minimas. Now the second derivative test fails for $(x,y)=(0, \pm \frac{1}{\sqrt 3})$ so what am I supposed to do?
The second derivative test fails for $(x,y)=(0,\pm 1/\sqrt{3})$. Call one of these points $(a_x,a_y)$. I would try to do this: take $f$ and replace $x$ with $a_x+t \cos(s)$ and $y$ with $a_y+t \sin(s)$. So you have $f(x(t),y(t))$.. I will call this function just $f(t)$. Expand $f(t)$ in series: for the point $a_x=0$ and $a_y=1/\sqrt{3}$ I obtain: $$ -(2/(3 \sqrt{3}))+\sqrt{3} \sin(s)^2 t^2 + \sin(s)^3 t^3 - (2 \cos(s)^4 t^4)/\sqrt{3}+O(t)^5 $$ In this way you see that the problem is for the direction $s=0$ (the $s=\pi$ is the same direction). Along this direction the first term that is non-zero is the fourth-order term, which is negative (so along the direction $s=0$ you have a maximum). For $s\neq0$, the first non-zero term is the second-order term, which is always positive.. so at the end I would call this a saddle point.