Singular Value Decomposition - Proof

Question

With singular value decomposition we can write the following: \begin{equation} A = U \Sigma V^{T} \end{equation} \begin{equation} U^{T}AV=U^{T}U\Sigma V^{T} V \end{equation} Since $U,V$ orthogonal, the above equation leads to the following: \begin{equation} \Sigma =U^{T}AV \end{equation} I've seen a proof that says the following \begin{equation} \Sigma^{-1}=V^TA^{-1}U \end{equation}

Can someone help with to understand how we ended up to the latter equation.

Answer 1

If all the matrices involved are square and invertible, we have $U^T = U^{-1}$ and $V^{T} = V^{-1}$, so $$ \Sigma^{-1} = (U^{-1}AV)^{-1} = V^{-1}A^{-1}U = V^TA^{-1}U $$ as desired.

Answer 2

Singular value decomposition

Begin with a nonzero matrix $\mathbf{A}\in\mathbb{C}^{m\times n}_{\rho}$ such that the matrix rank $\rho<m$ and $\rho<n$. The singular value decomposition, guaranteed to exist, is $$ \mathbf{A} = \mathbf{U} \, \Sigma \, \mathbf{V}^{*} = \left[ \begin{array}{cc} \color{blue}{\mathbf{U}_{\mathcal{R}}} & \color{red}{\mathbf{U}_{\mathcal{N}}} \end{array} \right] % \left[ \begin{array}{c} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % \left[ \begin{array}{c} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{V}_{\mathcal{N}}}^{*} \end{array} \right]. $$ The SVD provides an orthnormal basis for the four fundamental subspaces: $$ \begin{array}{ll} % matrix & subspace \\\hline % \color{blue}{\mathbf{U}_{\mathcal{R}}}\in\mathbb{C}^{m\times\rho} & \color{blue}{\mathcal{R}\left(\mathbf{A}\right)} \\ % \color{blue}{\mathbf{V}_{\mathcal{R}}}\in\mathbb{C}^{n\times\rho} & \color{blue}{\mathcal{R}\left(\mathbf{A}^{*}\right)} \\ % \color{red}{\mathbf{U}_{\mathcal{N}}}\in\mathbb{C}^{m\times m-\rho} & \color{red}{\mathcal{N}\left(\mathbf{A^{*}}\right)} \\ % \color{red}{\mathbf{V}_{\mathcal{N}}}\in\mathbb{C}^{n\times n-\rho} & \color{red}{\mathcal{N}\left(\mathbf{A}\right)} % \end{array} $$ There are $\rho$ singular values which are ordered and real: $$ \sigma_{1} \ge \sigma_{2} \ge \dots \ge \sigma_{\rho}>0. $$ These singular values for the diagonal matrix of singular values $$ \mathbf{S} = \text{diagonal} (\sigma_{1},\sigma_{1},\dots,\sigma_{\rho}) \in\mathbb{R}^{\rho\times\rho}. $$ The $\mathbf{S}$ matrix is embedded in the sabot matrix $\Sigma\in\mathbb{R}^{m\times n}$ whose shape insures conformability. $$ \Sigma = \left[ \begin{array}{c} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] = % Sigma \left[ \begin{array}{cccc|cc} \sigma_{1} & 0 & \dots & 0& 0& \dots & 0 \\ 0 & \sigma_{2} &&&&& \\ \vdots && \ddots &&\vdots&& \vdots\\ 0 & & & \sigma_{m} & 0 & \dots & 0 \\\hline 0 & \dots && 0 & 0 & \dots & 0 \\ \vdots & && \vdots & \vdots & \ddots & \vdots \\ 0 & \dots && 0 & 0 & \dots & 0 \\ \end{array} \right] $$

Moore-Penrose pseudoinverse

The components of the SVD can be used to construct the Moore-Penrose pseudoinverse: $$ \mathbf{A}^{\dagger} = \mathbf{V} \, \Sigma^{\dagger} \, \mathbf{U}^{*} = \left[ \begin{array}{cc} \color{blue}{\mathbf{V}_{\mathcal{R}}} & \color{red}{\mathbf{V}_{\mathcal{N}}} \end{array} \right] % \left[ \begin{array}{cc} \mathbf{S}^{-1} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % \left[ \begin{array}{c} \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{U}_{\mathcal{N}}}^{*} \end{array} \right]. $$ A common source of confusion comes from $\Sigma$ manipulations, so the main forms are shown here. Note that $\mathbf{S}^{\mathrm{T}} = \mathbf{S}$. Pay close attention to the shapes: $$ \begin{align} % \Sigma_{m\times n} &= \left[ \begin{array}{cc} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] , \quad % transpose \Sigma^{\mathrm{T}}_{n\times m} = \left[ \begin{array}{cc} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] , \quad % inverse \Sigma^{\dagger}_{n\times m} = \left[ \begin{array}{cc} \mathbf{S}^{-1} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % \end{align} $$ This provides a crisp construction of $\Sigma^{\dagger}$.

Derivation

To isolate the term $\Sigma^{\dagger}$, start with the definition of the psuedoinverse. $$ \begin{align} \mathbf{A}^{\dagger} &= \mathbf{V} \, \Sigma^{\dagger} \, \mathbf{U}^{*} \\ % \mathbf{V}^{*}\, \mathbf{A}^{\dagger} &= \mathbf{V}^{*}\, \mathbf{V} \, \Sigma^{\dagger} \, \mathbf{U}^{*} \\ % \mathbf{V}^{*}\mathbf{A}^{\dagger}\, \mathbf{U} &= \Sigma^{\dagger} \, \mathbf{U}^{*}\, \mathbf{U} \\ % \mathbf{V}^{*}\mathbf{A}^{\dagger}\, \mathbf{U} &= \Sigma^{\dagger} % \end{align} $$