Non-negativity of $x^2y^2(x^2+y^2-3)+1$

Question

1. Prove that polynomial $f(x,y)=x^2y^2(x^2+y^2-3)+1$ is non-negative.

2. Prove that $f(x,y)$ cannot be represented as the sum of squares of polynomials with real coefficients.

Would be grateful to see solution. I was thinking for couple days but unfortunately no results.

Answer 1

Considering the function

$$ f(x,y) = x^2 y^2 (x^2 + y^2 - 3) + 1 $$

we have a stationary point at $x = y = 1$

At this point the Hessian is

$$ H = \left( \begin{array}{cc} 4 & 2 \\ 2 & 4 \\ \end{array} \right) $$

which is definite positive then $f(x,y)$ has a minimum at $(1,1)$ and at this point $f(1,1) = 0$ hence

$$ f(x,y) \ge 0 $$

Answer 2

This is a famous example, due to Motzkin(1967). Meanwhile, the continuation is that the polynomial really is the sum of squares of rational functions. Proofs of the items in the question above can also be found in lectures posted online.