So first I calculated the gradient which was $\nabla f(x) = (64x_1^3 - 16x_1x_2, -8x_1^2 + 2x_2)^T.$ Then setting the equations in this equal to $0$ I solved for $x_2$ and got $x_2=4x_1^2$. Then calculated the Hessian matrix which I calculated to be \begin{equation*} \nabla ^2 f(x)=\left( \begin{array}{rr} 192x_1^2 -16x_2 & -16x_1 \\ -16x_1 & 2 \end{array} \right). \qquad \end{equation*} Then I plugged in $4x_1^2$ for $x_2$ and got \begin{equation*} \nabla ^2 f (x)= \left( \begin{array}{rr} 128x_1^2 & -64x_1^2 \\ -64x_1^2 & 2 \end{array} \right). \qquad \end{equation*} After this I'm stuck and don't really know how to figure out what the stationary points are. I know that there are stationary points whenever $x_2 = 4x_1^2$, but I'm not sure how to classify these as saddle points, local minimums, maximums or anything. Any help?
Thanks
$16x_1(4x_1^2−x_2) = 0\\ -2x_2(4x_1^2-x_2) = 0$
We have this parabola $4x_1^2 = x_2$ where the partial derivatives are $0.$
$(0,0)$ is on the parabola, so no need for a special call out.
$f(x_1,x_2) = 0$ for all points on the parabola.
$f(x_1,x_2) > 0$ for all points not on the parabola.
The parabola forms a sort of trough, and they are all minima.
As for your matrix:
$\begin{bmatrix} f_{xx} & f_{xy}\\ f_{xy}& f_{yy}\end{bmatrix}$
(by the way, I think you mistyped something) If the determinant of this matrix is greater than $0,$ you have a max or a min. If it is less than 0 you have a saddle. In this case it is exactly $0$ and no help at all.