We're studying semidefinite optimization in order to see if a polynomial can be written as a sum of squares. So to put it in a more math-appropriate form, for $p\in\mathbb{R}[x,y]$, we want to rewrite it of the form $p=\sum q_i^2 $ where $q_i\in\mathbb{R}[x,y]$, now we can rewrite $p = (x^2, xy, y^2)Q(x^2, xy, y^2)^T$ and if $Q$ is positive semidefinite, it can be written as $p = (x^2, xy, y^2)Q(x^2, xy, y^2)^T = (x^2, xy, y^2)LL^T(x^2, xy, y^2)^T = (x^2, xy, y^2)L(L(x^2, xy, y^2))^T$ which is a sum of squares.
Now, our lecturer said we would need to assure our diagonal elements are non-negative and (and this is where I've lost it) $det(Q)\geq 0$ in order for $Q$ to be PSD because then the eigenvalues are non-negative. But because $Q\in\mathbb{R}^{3\times3}$, we cant assume our matrix is PSD in that case because we could still have 2 negative eigenvalues and 1 non-negative.
Am I missing something?
You're correct: having nonnegative determinant and nonnegative diagonal elements is not enough to ensure that a $3 \times 3$ symmetric matrix is psd. For example,
$$ \pmatrix{0 & 1 & 0\cr 1 & 0 & 0\cr 0 & 0 & 0\cr}$$
You'll also want to require that the three $2 \times 2$ principal minors are nonnegative.