Given SVD of $X=USV^T$ where $S^T = [\Sigma, 0]$ is full rank, I'm struggling to show that $X(X^TX)^{-1}X^T=UU^T$ as stated in the first equation of this answer. My attempt is below:
$$X(X^TX)^{-1}X^T$$ $$USV^T(VS^TU^TUSV^T)^{-1}VS^TU^T$$ $$USV^T(VS^TS V^T)^{-1}VS^TU^T$$ $$USV^T(V \Sigma\Sigma V^T)^{-1}VS^TU^T$$ $$US(\Sigma \Sigma)^{-1}S^TU^T$$
I don't know if I've made a mistake somewhere above but if not, I can't figure out how to reduce $S(\Sigma \Sigma)^{-1}S^T$ into $I$.
On
I think you're not doing the inverse of the product well. Notice that
$$^(^^^)^{−1}^^ =\\ U S V^T V^{-T} S^{-1}U^{-1} U^{-T} S^{-T}V^{-1}VS^T U^T $$
Using this can you solve it? (Use associativity)
Edit: I am using the fact that $(AB)^{-1} = B^{-1} A^{-1}$.
Edit 2: Keep in mind that $U$ is orthogonal as well, so $UU^T = I$.
In the linked answer, the SVD was most likely taken as an “economic SVD” (a.k.a. “compact SVD”). If $X$ is an $n\times n$ matrix of rank $r$, this means $U$ is an $n\times r$ matrix with $r$ orthonormal columns, $S$ is an $r\times r$ positive diagonal matrix and $V$ is an $r\times r$ orthogonal matrix. Therefore $$ \begin{aligned} X(X^TX)^{-1}X^T &=USV^T(VSU^TUSV^T)^{-1}VSU^T\\ &=USV^T(VSI_rSV^T)^{-1}VSU^T\\ &=USV^T(VS^2V^T)^{-1}VSU^T\\ &=USV^T(VS^{-2}V^T)VSU^T\\ &=UU^T. \end{aligned} $$ Since $U$ is not necessarily a square matrix, $UU^T$ is in general not equal to $I_n$. That was probably why the author wrote $UU^T$ instead of $I$.