I'm working on a problem, and to finish it I need to show that
$$\mu^T (I-X(X^TX)^{-1}X^T)\mu \geq 0$$
But I'm stuck at showing this.
My first thought was to write this as trace
$$tr(\mu^T\mu) \geq tr(\mu^T X(X^TX)^{-1}X^T \mu)$$
and try to use the idempotence of $X(X^TX)^{-1}X^T$ and the fact its rank is $r(X) \leq n$. However I don't really see how that helps here.
Alternatively, I thought I could use the formula for the expected value of a quadratic form, which involves terms like the traces involved. But again, not much luck.
Any advice?
The matrix $X(X^TX)^{-1}X^T$ is not only idempotent but also symmetric, hence an orthogonal projection. The same is true for $I -X(X^TX)^{-1}X^T$: it is symmetric and idempotent. Then $$ \mu^T( I -X(X^TX)^{-1}X^T)\mu = \mu^T( I -X(X^TX)^{-1}X^T)^2\mu = \|( I -X(X^TX)^{-1}X^T)\mu\|^2 \ge0. $$