Showing the matrix is non-negative definite

41 Views Asked by At

Let $X$ be full column rank.

I am trying to show that the matrix

$$(X^TX)^{-1}(X\beta)^T(X\beta)-\beta\beta^T$$

is non-negative definite.

Here, $\beta$ is a parameter vector with intercept.

To show it's non-negative definite, I know I need to use the following fact that if $v$ is a vector, then:

$$I-vv^T \geq 0 \iff v^Tv \leq 1$$

But I'm not sure how to get to the point where we can express it in this way. If $z$ is any vector, then I need to show that

$$z^T [(X^TX)^{-1}\beta^T X^T X\beta-\beta\beta^T]z \geq 0$$

But I don't see how I can rearrange this to use the above lemma. Since $\beta^T X^T X\beta$ is a scalar, I thought of using trace and cyclic property to write it as $X^TX\beta\beta^T$, but then ofcourse $(X^TX)^{-1}$ is stuck outside the trace, and I am stuck.