I'm trying to prove that the function $f: \mathbb{R} ^2 \to \mathbb{R}$ such that $f(x,y)=3x^4-4x^2y+y^2$ has no local minimum in any disk, but my proof is long and very inefficient. Any ideas?
I started my proof noticing that $f(x,y)=(3x^2-y)(x^2-y)$
Then $f(x,y) < 0$ if $3x^2>y>x^2$ and $f(x,y) \geq 0$ otherwise.
In the case where $3x^2>y>x^2$:
Let $S=\{(x,y) \in \mathbb{R} ^2 | 3x^2>y>x^2\}$. Let $\delta>0$ and $D=B_\delta (0,0)-\{(0,0)\}$ an arbitrary disk.
If $u \in S \cup D$, then I tried to proof that always exists $v \in S \cup D$ such that $f(u) > f(v)$ and in this way, there's no local minimum in the disk. But my proof is very very long.
And I didn't do the proof of the second case (i.e., $u \notin S$).
So, am I in the correct way, or there's an easier way for doing this?
Given
$f(x,y)=3x^4-4x^2y+y^2,\tag 1$
we may investigate the existence and locaion of critical points by first taking
$f_x = 12x^3 - 8xy, \tag 2$
$f_y = -4x^2 + 2y, \tag 3$
we see that
$f_y = 0 \Longrightarrow y = 2x^2, \tag 4$
and then
$f_x = 0 \Longrightarrow 12x^3 = 8xy = 8x(2x^2) = 16x^4 \Longrightarrow 4x^3 = 0 \Longrightarrow x = 0; \tag 5$
it follows from these two statements that the only critical point of $f$ occurs at
$(x, y) = (0, 0); \tag 6$
since relative minima are critical, we need merely investigate the nature of this the sole critical point of $f$; a first step is to compute the second derivatives; we have
$f_{xx} = 36x^2 - 8y, \tag 7$
$f_{yy} = 2, \tag 8$
$f_{xy} = f_{yx} = -8x; \tag 8$
we may now form the Hessian of $f$:
$H[f](x, y) = \begin{bmatrix} f_{xx} & f_{xy} \\ f_{yx} & f_{yy} \end{bmatrix} = \begin{bmatrix} 36x^2 - 8y & -8x \\ -8x & 2 \end{bmatrix}; \tag 9$
at our critical point $(0, 0)$ this becomes
$H[f](0, 0) = \begin{bmatrix} f_{xx} & f_{xy} \\ f_{yx} & f_{yy} \end{bmatrix} = \begin{bmatrix} 0 & 0 \\ 0 & 2 \end{bmatrix}; \tag{10}$
since $H[f](0, 0)$ is singular, we see that the critical point $(0, 0)$ is degenerate, and we can't directly assert, based upon $H[f](0, 0)$, whether the point is a local minimum or not, or indeed just what the topological characteristics of the level sets of $f$ in a neighborhood of $(0, 0)$ might be. Therefore a change in direction is mandated.
We observe that
$f(x, y) > 0 \tag{11}$
on both the $x$- and $y$-axes; that is
$f(x, 0), \; f(0, y) > 0, \; \forall 0 \ne x, y \in \Bbb R; \tag{12}$
this fact, together with
$f(0, 0) = 0 \tag{13}$
suggests that we attempt to find the curves $y(x)$ along which
$f(x, y(x)) = 0, \tag{14}$
since if $f(x, y)$ undergoes a change sign from positive to negative, which must occur arbitrarily close to $(0, 0)$ if this point is not a local minimum of $f$, it must occur as one of these curves $y(x)$ is crossed. We thus write the condition (14) as a quadratic equation in $y$ with coefficients functions of $x$:
$y^2 - 4x^2 y + 3x^4 = 0;, \tag{15}$
which we may solve via the quadratic formula:
$y = \dfrac{4x^2 \pm \sqrt{16x^4 - 12x^4}}{2} = \dfrac{4x^2 \pm \sqrt{4x^4}}{2} = \dfrac{4x^2 \pm 2x^2}{2} = 3x^2, x^2; \tag{16}$
it follows that $f(x, y) = 0$ along the curves
$y = x^2, 3x^2, \tag{17}$
this suggests we investigate the sign of $f(x, y)$ along
$y = 2x^2, \tag{18}$
which lies between the two curves (17) in the $xy$-plane; and we have
$f(x, 2x^2) = (2x^2)^2 - 4x^2(2x^2) + 3x^4 = 4x^4 - 8x^4 + 3x^4 = -x^4 < 0 \; \text{for} \ x \ne 0; \tag{19}$
this shows $f(x, y) < 0$ arbitrarily close to $(0, 0)$, so the origin cannot be a local minimum of $f(x, y)$; since $(0, 0)$ is the only critical point of $f(x, y)$, this function has no local minimum in the plane, hence not in any disk. $OE\Delta$.
The region where $f(x, y) < 0$ is apparently squeezed between the two curves $y = x^2$ and $y = 3x^2$, and narrows to a sort of cusp as $x \to 0$; so it is very tiny, a mere sliver near the origin; hence, hard to find.
But note the methods here are primarily heuristic in nature; the solution involved some (hopefully) well-guided guesswork.