I'm going over some old assignments from a couple terms ago and have come across a problem from my variational principals module.
I looked at the function in the hint and noticed that for some points $f(x,y)<0$ but $f(0,0)=0.$ So my issue is in my understanding, particularly what is meant by 'function obtained by restricting $f(x,y)$ onto a straight line passing through the origin'. Does this mean that we take an line going through the origin in $\mathbb{R}^3$ or a line in the $x y$ plane going through the origin and consider the function $g(x)=f(x,kx)$ for some $k\in{\mathbb{R}}$. Or if neither of these, then what? Clearly the hint is to suggest a counter example but this function is not minimum at the origin when you restrict it as described (unless my understanding of the restriction is incorrect which is the most likely case).
Recall the definition of a restriction of a function, for instance, from Wikipedia. Let $f$ be a function from a set $E$ to a set $F$. If a set $A$ is a subset of $E$, then the restriction of $f$ to $A$ is the function $f|A:A\to F$ given by $f|_A(x) = f(x)$ for (each) $x$ in $A$.
That is, we have to consider restriction of the function $f(x,y)$ onto straight lines of its domain $\Bbb R^2$ passing through the origin. Such straight lines $A$ are given by a linear equation $x=0$ or $y=kx$ for some real $k$. If $A=\{(x,y):x=0\}$ then $f(x,y)|_A=y^4$ and this function attains the global minimum at $(0,0)$. If $A=\{(x,y):y=kx\}$ then $f(x,y)|_A=(x-k^2x^2)(2x-k^2x^2)=x^2(1-k^2x)(2-k^2x)$. So $f(0,0)=0$, but $f|A$ not always attains its minimum at $(0,0)$. For instance, for $k\ne 0$ and $x=\tfrac{3}{2k^2}$ we have $f(x,kx)<0$.