Let $f:\mathbb{R^2} \to \mathbb{R}$ be defined by
$$f(x,y)=x^6-2x^2y-x^4y+2y^2$$
Then which of following is true
$1).$ $f$ has a saddle point at origin
$2)$. The origin is not a critical point $f$
Solution I tried-Here i find the $f_{xx}(0,0),f_{yy}(0,0)$ and $f_{xy}(0,0)$ all i get is $0$ $i.e$ the value of $D$
$$D=f_{xx}f_{yy}-f_{xy}^2$$ i get this value zero and according to rule we cant say anything about this , i have no idea how to proccede further
Please Help
$(0,0)$ is a critical point because $f_x(0,0)=f_y(0,0)=0$. As you said, the second partial derivative test fails. To see that it's a saddle point, note we have the factorization $f(x,y)=(x^4-2y)(x^2-y)$. Then $f(x,x^3)=x^4(x-2)(1-x)x$, so $f(x,x^3)>0$ for $-1<x<0$ and $f(x,x^3)<0$ for $0<x<1$. That is, you have both negative and positive values around $(0,0)$ and $f(0,0)=0$.
As an alternative, $x=0$ is a local minimum point for $f(x,x)=x^2(x^3-2)(x-1)$ and a local maximum point for $f\left(x,\frac{x^2}{2}\right)=\frac{x^4}{2}(x^2-1)$.