I would like to prove $r(X) = r(P_{X}) = \text{tr}(P_X)$, $r$ denoting the rank, $X \in M_{n \times p}(\mathbb{R})$, and $$P_{X} = X(X^{T}X)^{-}X^{T}$$ where $(X^{T}X)^{-}$ is a generalized inverse of $X^{T}X$.
I don't believe I know the machinery to prove this. But here's what I do know that might possibly be relevant.:
- $P_X$ is symmetric. So therefore, the trace of $P_X$ is the sum of its eigenvalues. This suggests to me that the sum of its eigenvalues is also its rank (for whatever reason).
- I have a feeling that the Singular Value Decomposition is involved. But I have no idea how to use this nor am I familiar with its properties (I am completely self-taught when it comes to SVD).
An explanation of how to pursue this proof would be appreciated.
Let $X = U\Sigma V^T$ be the SVD of $X$, where $\Sigma$ is $n\times p$. Then
$$ X^TX = V\Sigma^T\Sigma V^T $$
is the SVD of $X^TX$. Hence, $(X^TX)^- = V(\Sigma^T\Sigma)^-V^T$. It follows that
$$ P_X = X(X^TX)^-X^T = U\Sigma(\Sigma^T\Sigma)^-\Sigma^T U^T = U \left [ \begin{array}{cc} I_{r\times r} & 0\\ 0 & 0\\ \end{array} \right ] U^T $$
is the SVD of $P_X$. The desired equalities should now follow readily.