Given the function $f(x,y)=x^3+4y^2-4xy$ to be evaluated over the set $E={(x,y) \in R^2: 0\leq y \leq 3x/4}$, I'm asked to determinate $F(E)$.
I've noticed that the function in continuous, and the set given is not bounded. I thought that if I can prove that the function is coercive, as a consequence of Weiestrass th. I got that on an unbounded, closed set $f$ has a miniminum. Or otherways is f is anticoercive has a maximum.
But I don't know how to prove it, just using intuition I got that the sup bounf of $f(E)$ is $+\infty$.
I also thought that if I suppose the previous Lemma is working I can study the stationary points of the function ($\nabla f=0$) and I got $(0,0); (2/3;1/3)$ for the second point $f(2/3;1/3)=-4/27$ which is the global minimum.
Honestly this is just an attempt and I have no theory supporting my works so I'm looking for any kind of help or rigorous method able to help me treating this kind of problem.
Is it possible to prove that the function is coercive?
Considering the critical points is not a bad idea, but it's unlikely to solve the problem because the function might approach its max and min near the boundary of $E$. So it's better to bring the geometry of $E$ into play early on.
The set $E$ is the union of half-lines $\{(x,kx):x\ge 0\}$ where $k$ ranges from $0$ to $3/4$. Therefore, its image under $f$ is the union of the images of these lines. The reason to look at half-lines is that $$f(x,kx) = x^3 - 4k(1-k ) x^2$$ is a simple single-variable function; call it $g_k$. Observe that every $g_k$ is unbounded from above; hence $\sup f=+\infty$. Also, $g_{1/2}$ is the smallest of these functions (meaning $g_{1/2}\le g_k$ everywhere), since $k(1-k)$ is maximized by $k=1/2$. So, $\inf f$ is the minimum of $g_{1/2}(x) = x^3-x^2$ on $[0,\infty)$, which is easy to find.