Uniqueness of Singular Values

Question

Given a matrix A, one inductively constructs (and thus proving its very existence) the singular value decomposition as follows: take $ \sigma_{1}=||A||_{2} $, and consider a couple of vectors such that \begin{equation} A\textbf{v}_{1} = \sigma_{1} \textbf{u}_{1} \end{equation} Now extend to two orhogonal bases and write down the first step: \begin{equation} U_{1}^{T} A V_{1} = S = \begin{bmatrix} \sigma_{1} & \textbf{0}^{T} \\ \textbf{0} & A_{1} \end{bmatrix} \end{equation} Now take $A_{1}$, and repeat inductively, costructing \begin{equation} U_{2}^{T} A_{1} V_{2} = S_{2} = \begin{bmatrix} \sigma_{2} & \textbf{0}^{T} \\ \textbf{0} & A_{2} \end{bmatrix} \end{equation} which gives rise to second singular value, by selecting $\sigma_{2}=||A_{1}||$.

My question is: suppose we choose another couple of extensions to orthogonal bases, call them $\hat{U_{1}},\hat{V_{1}}$. Doing so, we'll obtain another matrix $\hat{A_{1}}$: who guarantees this $\hat{A_{1}}$ has the same norm of ${A_{1}}$, thus giving the same second singular value $\sigma_{2}$?

Thanks.

Answer 1

Note that if the columns of $U$ and $\hat{U}$ span the same subspace, then $\hat{U}=UM$ for some nonsingular $M$. If the columns of both $U$ and $\hat{U}$ are orthonormal, the matrix $M$ is orthogonal because $I=\hat{U}^T\hat{U}=M^TU^TUM=M^TM$.

So, if $U_1=[u_1,\tilde{U}_1]$ and $\hat{U}_1=[u_1,\tilde{\hat{U_1}}]$, then $\hat{U}_1=[u_1,\tilde{U}_1]M$ for some orthogonal matrix $$M=\begin{bmatrix}1&0\\0&\tilde{M}\end{bmatrix}.$$

Similarly, $\hat{V}_1=V_1N$ for an orthogonal matrix $$N=\begin{bmatrix}1 & 0\\0&\tilde{N}\end{bmatrix}.$$

Therefore, $$ \hat{A}_1=\tilde{\hat{U}}_1^TA\tilde{\hat{V}}_1=M^T\tilde{U}_1^TA\tilde{V}_1N=\tilde{M}^TA_1\tilde{N}. $$

The matrices $A_1$ and $\hat{A}_1$ have the same 2-norm since the 2-norm is orthogonally invariant, that is, $\|A_1\|=\|PA_1Q\|$ for any square orthogonal matrices $P$ and $Q$ because $\|PA_1Q\|^2=\rho((PA_1Q)^T(PA_1Q))=\rho(Q^TA_1^TA_1Q)=\rho(A_1^TA_1)=\|A_1\|^2$ and hence in the induction step the $\sigma_2$ is same no matter what orthonormal bases of the orthogonal complements of $u_1$ and $v_1$ you choose.

Answer 2

Note that $$ \widehat U_1^T U_1 \pmatrix{\sigma_1&0^T\\0&A_1}V_1^T\widehat V_1 = \pmatrix{\sigma_1&0^T\\0&\widehat A_1} $$ Break the unitary matrices into block matrices to see that $\widehat A_1 = U A_1 V^T$ for unitary matrices $U,V$.