Let $f: \mathbb R^{2}\to\mathbb R$ and $f(x,y)=(x^2-1)(y-1)^2$. Find all the extrema and saddle points.
Idea: Setting $\nabla f(x,y)=0$, we get:
i) $2x(y-1)^2=0$
ii) $(x^2-1)2(y-1)=0$
Therefore in order for i), ii) to be satisfied it follows in all cases of $x\in\mathbb R$ , $y=1$:
Thus our critical points would be $\{(x,1):x\in \mathbb R\}$
So now looking at the Hessean Matrix:
$(D^2f)(x,y)=\begin{pmatrix} 2(y-1)^2 & 4x(y-1) \\ 4x(y-1) & 2(x^2-1) \end{pmatrix}$
And for:
$(D^2f)(x,1)=\begin{pmatrix} 0 & 0 \\ 0 & 2(x^2-1) \end{pmatrix}$
And all of these are of indeterminate form, thus we need to look at the function itself:
for $|x|<1$, we have $f(x,y)\leq0$ and for $|x|>1$ we have $f(x,y)\geq0$ and in all cases of the above $f(x,y)=0\iff y=1$ or $x=\pm1$
but our only critical values are found in $M:=\{(x,1):x\in \mathbb R\}$, and thus all critical points $M$ are saddle points (I am not too sure about my reasoning here...)
If $|x|<1$, then $f(x,y)\leqslant0$ near a point of the type $(x,1)$ and therefore, $(x,1)$ is a local maximum.
And if $|x|>1$, then $f(x,y)\geqslant0$ near a point of the type $(x,1)$ and therefore, $(x,1)$ is a local minimum.
Finally, if $x=\pm1$, there are points as close as you wish of $(\pm1,1)$ where $f$ takes values greater than $0$ and there are points as close as you wish of $(\pm1,1)$ where $f$ takes values smaller than $0$. Therefore, $(\pm1,1)$ are saddle points.