Let $y=(A^TA)^{-1}A^Tb$ where $A$ is an $m\times n$ matrix with full column rank and $b$ is a vector. Express $y$ in terms of the left and right singular vectors of $A$, $u_i\in \mathbb R^{m\times 1}$ and $v_i\in \mathbb R^{n\times 1}$, respectively, $b$, and the singular values $\{\sigma_1,\ldots,\sigma_n\}$.
$$y=V\begin{bmatrix} \Sigma^{-1} & 0 \\ \end{bmatrix}U^Tb$$
- How do I expand this product in terms of $u_i\in \mathbb R^{m\times 1},\,v_i\in \mathbb R^{n\times 1}$, and $\sigma_i$?
You have
$$ y = V \begin{bmatrix} \Sigma^{-1} & 0 \end{bmatrix} \begin{bmatrix} {U_{1:n}}^Tb\\ {U_{n+1:m}}^Tb \end{bmatrix} = V\Sigma^{-1}{U_{1:n}}^Tb = \sum_{i=1}^n\frac{1}{\sigma_i}v_iu_i^Tb $$