I have the function $f(x,y)=x^2+y^4+xy$ and I need to see if it is convex. So I take the hessian matrix: $$\nabla ^2f(x,y)=\begin{pmatrix} 2 & 1\\ 1 & 12y^2 \end{pmatrix}=A$$
And from here I find the principal minors: $$Det(A[[1],[1]])=2$$ $$Det(A[\{2\},\{2\}])=12y^2$$ $$Det(A[[2],[2]])=24y^2-1$$
And from those I see that they are not always non negative, because $y=0 \Rightarrow 24y^2-1=-1$
So therefore the function is not convex, is this correct?
You are correct. Note that for $(x,y)$ near the origin, the function looks like $x^2+xy=(x+y/2)^2-y^2/4$, whose graph is a saddle surface.