I'm trying to prove the following statement. It seems pretty interesting but I have no idea of how to prove it:
Let $X$ be a $n\times p$ matrix, rank deficient. We can continuously add row to $X$ which is linearly independent with the row space of $X$ until full rank. That it, we obtain $\begin{pmatrix} X\\ A\end{pmatrix}$, where $X$ and $A$ are linearly independent. Prove that: $$(X^TX+A^TA)^{-1}$$ is an generalized inverse of $X^TX$, i.e, $(X^TX)(X^TX+A^TA)^{-1}(X^TX)=X^TX$
Is it really correct? I try a lot of examples but can't find any counter-examples...
Let $v$ be an arbitrary vector. We will show that $X^TX(X^TX + A^TA)^{-1}X^TXv = X^TXv$. It will then follow that $X^TX(X^TX + A^TA)^{-1}X^TX = X^TX$.
Labeling, $v' = X^TXv$ we have that $v'$ is in the span of the rows of $X$. This means that if $(X^TX + A^TA)^{-1}v = b$ we have $v' = X^TXb$ (take a moment to show this yourself, but it is in the spoiler).
This shows that
$$X^TX(X^TX + A^TA)^{-1}(X^TX)v = X^TX(X^TX + A^TA)^{-1}v' = X^TXb = v' = X^TXv$$