When I try to factor the quadratic form, I end up with
$$6x^2+4y^2+2z^2+4xz-4yz = 2((x+z)^2+2x^2+(y-z)^2-z^2)$$
which does not ensure that $f(x,y,z) \geq 0$ for all $x, y, z \geq 0$ since the $z$ term is negative. How should these kinds of problems be tackled?
On
There's an error in your computation - the $y^2$ term doesn't match on both sides of the equation.
Having said that, you can still argue that the RHS is positive because $(x+z)^2 \geq z^2$ if $x\geq0$.
On
We need to prove that $$z^2+2(x-y)z+3x^2+2y^2\geq0$$ for which it's enough to prove that $$(x-y)^2-(3x^2+2y^2)\leq0$$ or $$2x^2+2xy+y^2\geq0$$ or $$x^2+(x+y)^2\geq0,$$ which is obvious.
Done!
On
I would like to add another solution that works in this setting, and also somewhat more generally, even though it's overkill for your specific question.
This thing is smooth, as it is just a polynomial, so we can look for extrema, and then test if they are maxima or minima. These happen when all partials vanish simultaneously, since there is no boundary to check.
$$\frac{\partial f}{\partial x} = 12x + 4z$$
$$\frac{\partial f}{\partial y} = 8y - 4z$$
$$\frac{\partial f}{\partial z} = 4z + 4x - 4y$$
And we want these to simultaneously vanish. We can see from the third line that this happens when $ y= x + z$, so then the second line gives $8(x+z) - 4z = 0 \implies8x + 4z = 0$ This implies $z = -2x$. Then the first line gives $x = 0$, so we recover $z = 0$, and these give $y = 0$. So the only critical point is $x = y = z = 0$. Indeed, this could be seen from the general fact that we have a homogeneous linear system.
Now we need to compute the Hessian matrix, to use the Second Partials Test.
$$ \begin{bmatrix} \frac{\partial^2f}{\partial x^2} & \frac{\partial^2f}{\partial x \ \partial y} & \frac{\partial^2f}{\partial x \ \partial z} \\ \frac{\partial^2f}{\partial y \ \partial x} & \frac{\partial^2f}{\partial y^2} & \frac{\partial^2f}{\partial y\ \partial z} \\ \frac{\partial^2f}{\partial z \ \partial x} & \frac{\partial^2f}{\partial z \ \partial y} & \frac{\partial^2f}{\partial z^2} \\ \end{bmatrix} $$
These are easy to compute, noting that since everything is smooth, they are $C^2$, and this is more than enough for mixed partials to commute, so we only compute half the entries:
$$\frac{\partial f^2}{\partial x^2} = 12 \ \ \ \ \frac{\partial^2 f}{\partial x \ \partial y} = 0 \ \ \ \ \frac{\partial^2f}{\partial x \ \partial z} = 4$$
$$\frac{\partial^2 f}{\partial y^2} = 8 \ \ \ \ \frac{\partial^2f}{\partial y \ \partial z} = -4$$
$$\frac{\partial^2 f}{\partial z^2} = 4$$
So putting this all together we get the matrix:
$$ \begin{bmatrix} 12 & 0 & 4 \\ 0 & 8 & -4 \\ 4 & -4 & 4 \\ \end{bmatrix} $$
We will have an extrema if this matrix is definite, and in particular, a minimum if it is positive definite. The characteristic polynomial of this guy is $-λ^3+24λ^2-144λ+64=0$,which feeding into any root-finder will tell you has all positive roots. This proves you have a minimum at this point.
Since clearly for large $x,y,z$, your function is getting larger, there is no worry of decay at infinity. This is the only minimum.
On
Your quadratic form is positive definite. I do not know what the eigenvalues of the matrix are. This method is the same as "completing the square." The diagonal entries of the diagonal matrix $D$ are all positive.
$$ P^T H P = D $$ $$\left( \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ - \frac{ 1 }{ 3 } & \frac{ 1 }{ 2 } & 1 \\ \end{array} \right) \left( \begin{array}{rrr} 6 & 0 & 2 \\ 0 & 4 & - 2 \\ 2 & - 2 & 2 \\ \end{array} \right) \left( \begin{array}{rrr} 1 & 0 & - \frac{ 1 }{ 3 } \\ 0 & 1 & \frac{ 1 }{ 2 } \\ 0 & 0 & 1 \\ \end{array} \right) = \left( \begin{array}{rrr} 6 & 0 & 0 \\ 0 & 4 & 0 \\ 0 & 0 & \frac{ 1 }{ 3 } \\ \end{array} \right) $$ $$ Q^T D Q = H $$ $$\left( \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ \frac{ 1 }{ 3 } & - \frac{ 1 }{ 2 } & 1 \\ \end{array} \right) \left( \begin{array}{rrr} 6 & 0 & 0 \\ 0 & 4 & 0 \\ 0 & 0 & \frac{ 1 }{ 3 } \\ \end{array} \right) \left( \begin{array}{rrr} 1 & 0 & \frac{ 1 }{ 3 } \\ 0 & 1 & - \frac{ 1 }{ 2 } \\ 0 & 0 & 1 \\ \end{array} \right) = \left( \begin{array}{rrr} 6 & 0 & 2 \\ 0 & 4 & - 2 \\ 2 & - 2 & 2 \\ \end{array} \right) $$
As an alternative at completing the square you can look at the matrix associated to the quadratic form:
$$x^TAx$$
$$A=\begin{bmatrix} 6 & 0 & 2 \\ 0 & 4 & -2 \\ 2 & -2 & 2 \\ \end{bmatrix}$$
and note that its signature is $(n_0=0,n_+=3,n_-=0)$thus the quadratic form is definite positive.