I've asked this question before, but the answers just made me even more confused.
Consider Rosenbrock’s (modified) Banana Function
$$f(x,y) = 36(y − x^2)^2 + (x − 1)^2$$
Show that the Hessian of this function is positive semidefinite in the region $y \leq x^2 + \frac{1}{72}$ . Explain why the function is not convex in this region.
I have found the Hessian to be
$$H(x,y) = \begin{bmatrix} -144(y-x^2)+288x^2+2 & -144x\\ -144x & 72\end{bmatrix}$$
I had the idea to apply inequality at the beginning to $f(x,y)$, which gave eigenvalues $0$ and $2$, showing positive semidefiniteness. I have also tried applying on the Hessian only, which gave the result $0$ and $72+288x^2$, which also shows positive semidefiniteness. Apparently this doesn't work. I have absolutely no idea how to explain why it isnt convex in the region though, as we have only been taught that positive semidefiniteness implies convexity.
We haven't had anything similiar in class. We aren't even past quadratic, let alone quartics, and when I tried to simply sub in the inequality, someone said that you cannot do that. I do not know how to answer this.