I should show that in diagonalizing matrix $B =[0, D^T; D, 0]$, I can obtain all the information needed for SVD of D.

Question

I am looking for a relation between the eigen vectors of B and the singular vectors of D where B = \begin{pmatrix} 0 & D^T \\ D & 0 \end{pmatrix} and D is n*d. I know that I should find the eigen vectors of B and then a relationship comes up, but I'm confused with calculating the determinant of B. I hope you could help me with the whole solution, thanks!

Answer 1

We are given that $D$ has shape $n \times d$ and that $B$ is the block-matrix $$ B = \pmatrix{0 & D^T\\D& 0}. $$ Because $B$ is symmetric, it has real eigenvalues. Suppose that the block-vector $x = (u,v)$ (with $u \in \Bbb R^d$ and $v \in \Bbb R^n$) is an eigenvector associated with a non-zero eigenvalue $\lambda \in \Bbb R$. We have \begin{align} Bx &= \lambda x \implies \pmatrix{0&D^T\\ D&0}\pmatrix{u\\v} = \pmatrix{\lambda u\\ \lambda v} \implies \pmatrix{Dv\\D^Tu} = \pmatrix{\lambda u\\ \lambda v} \\&\implies \begin{cases} Dv = \lambda u\\ D^Tu = \lambda v \end{cases} \end{align} Consider the vector $y = (u,-v)$. We have $$ By = \pmatrix{-Dv\\ D^Tu} = \pmatrix{-\lambda u\\ \lambda v} = -\lambda \pmatrix{u\\-v} = -\lambda y. $$ So, we can conclude that the non-zero eigenvalues of $B$ come in pairs of the form $\pm \lambda$. Note also that because $B$ is symmetric, eigenvectors associated with distinct eigenvalues must be orthogonal. Thus, we have $$ x^Ty = 0 \implies u^Tu - v^Tv = 0 \implies u^Tu = v^Tv. $$ That is, $\|u\| = \|v\|$. If $x$ is a unit-vector, then $\|u\| = \|v\| = 1/\sqrt{2}$.

Because $B$ is a symmetric matrix, it has an orthonormal set of eigenvectors. So, suppose that $\lambda_1 \geq \lambda_2 \geq \cdots \geq \lambda_r > 0$ are the positive eigenvalues of $B$. For each $i$, let $x_i = \frac 1{\sqrt{2}}(u_i,v_i)$ be an eigenvector associated with $\lambda_i$ such that the vectors $x_1,\dots,x_r$ are orthonormal. By the discussion above, we have $\|u_i\| = \|v_i\| = 1$. For each $i$, let $y_i = \frac 1{\sqrt{2}}(u_i,-v_i)$.

For any $i \neq j$, we have already stated that $x_i,x_j$ are orthogonal, and $x_i,y_j$ must also be orthogonal because the are eigenvectors associated with distinct eigenvalues. We note that $$ x_i^Tx_j = 0 \implies u_i^Tu_j + v_i^Tv_j = 0\\ x_i^Ty_j = 0 \implies u_i^Tu_j - v_i^Tv_j = 0 $$ Conclude that $u_i^Tu_j = 0$ and $v_i^Tv_j = 0$. With that, we can conclude that the sets $\{u_1,\dots,u_r\}$ and $\{v_1,\dots,v_r\}$ are orthonormal sets of vectors such that for all $i$, we have $$ Dv_i = \lambda_i u_i, \quad D^Tu_i = \lambda_i v_i. $$ With this, we may conclude that $\lambda_1,\dots,\lambda_r$ are the non-zero singular values of $D$, that $u_1,\dots,u_r$ are the associated left singular vectors, and $v_1,\dots, v_r$ are the associated right singular vectors.

Completing each orthonormal set into bases $u_1,\dots,u_n$ and $v_1,\dots,v_d$ produces a complete singular value decomposition of $D$.

---

Note 1: In your question, you ask about calculating the deteriminant of $B$, presumably for the purpose of computing a characteristic polynomial. As it turns out, we didn't need determinants. However, if you are interested in directly computing a characteristic polynomial anyway, you could use the Schur complement to find that $$ \det(B - \lambda I) = \det \pmatrix{-\lambda I & D^T\\ D & -\lambda I} = \det(-\lambda I) \det(-\lambda I - D(-\lambda I)^{-1}D^T)\\ = (-1)^{n + d} \lambda^d \det(\lambda I - \lambda^{-1}DD^T)\\ = (-1)^{n+d} \lambda^{d-n} \det(\lambda^2 I - DD^T). $$ One could apply the same steps applying the Schur complement of the bottom-right block or apply Sylvester's determinant identity to the above result to get $$ \det(B - \lambda I) = (-1)^{n+d} \lambda^{n-d} \det(\lambda^2 I - D^TD). $$ We could use either of these results to characterize the eigenvalues of $B$ as follows: if (and only if) $\lambda$ is a non-zero eigenvalue of $B$, it is a solution to $\det(\lambda^2 I - DD^T) = 0$, which is to say that $\lambda^2$ is an eigenvalue of $DD^T$, which is to say that $\lambda = \sigma$ or $\lambda = -\sigma$ for some (non-zero) singular value $\sigma$ of $D$.

---

Note 2: In the above, I thoroughly answer the following question – how exactly (and to what extent) can a spectral decomposition of $B$ be used to obtain the singular value decomposition of $D$? What I show is that the singular values and singular vectors can be directly obtained from the non-zero eigenvalues of $B$ and their associated eigenvectors.

What I don't say explicitly is that the singular vectors corresponding to the zero singular value cannot be obtained from the spectral decomposition of $B$. In particular, we know that $x_1,\dots,x_r$ (which can be taken from the spectral decomposition) and $y_1,\dots,y_r$ (constructed using the $x_i$) form an orthonormal set of eigenvectors for $B$. We know that whatever eigenvectors are found in the spectral decomposition of $B$ corresponding to zero, say $z_1,\dots,z_{n+d-2 \operatorname{rank}(D)}$, span the orthogonal complement to $\operatorname{span}\{x_1,\dots,x_r,y_1,\dots,y_r\}$. We do not know a priori that when the $z_i$ are broken into components $z_i = (p_i,q_i)/\sqrt{2}$, the sets $\{x_1,\dots,x_r,p_1,\dots\}$ and $\{y_1,\dots,y_r,q_1,\dots\}$ form orthonormal bases.