How is $X(X^{\prime}X)^{-}X^{\prime}$ symmetric?

Question

For a matrix $X$, a generalized inverse of $X$ is any matrix $Y$ such that $XYX = X$. We use $X^{-}$ to indicate a generalized inverse of $X$.

Suppose $X$ is a matrix. $X^{\prime}$ denotes the transpose of $X$. Then clearly $X^{\prime}X$ is symmetric and has a generalized inverse that is symmetric [say $(X^{\prime}X)^{-}$] by a theorem I previously proved. This I follow.

This textbook I have then starts saying $X(X^{\prime}X)^{-}X^{\prime}$ is symmetric without any discussion as to why, and no previous theorems or lemmas seem to hint at this. Why is this the case?

[I am sorry if this isn't enough information to show my effort, but I am extremely lost as to why to think this is true.]

Answer 1

This question was more obvious than I would've liked it to be, hah.

We have $$\left[X(X^{\prime}X)^{-}X^{\prime}\right]^{\prime} = \left(X^{\prime}\right)^{\prime}\left[(X^{\prime}X)^{-}\right]^{\prime}X^{\prime} = X\left[(X^{\prime}X)^{-}\right]^{\prime}X^{\prime}$$ and see this comment.

Answer 2

By a unitary changes of bases (note the plural form here --- $X$ in general is rectangular and we view it as a linear transformation between two vector spaces of possibly different dimensions) or singular value decomposition, you may assume that $X=\pmatrix{D&0\\ 0&0}$ where $D$ is a real invertible diagonal matrix. Then $X'X=\pmatrix{D^2&0\\ 0&0}$. It follows that any generalised inverse of $X'X$ must be in the form of $\pmatrix{D^{-2}&\ast\\ \ast&\ast}$. Hence $$ X(X'X)^-X' = \pmatrix{I&0\\ 0&0}, $$ which is Hermitian. So it is Hermitian with respect to the original bases too, and it is real symmetric when $X$ is real.