Why is $S$ in SVD a vector instead of a matrix?

Question

I know that when applying SVD on a matrix (m * n) I should have these three outputs:

S: m × n diagonal matrix with non negative numbers
U: m × m orthogonal matrix
V: n × n orthogonal matrix

but when using R statistical package. I got for S a vector instead of a matrix:

look please:

this is the matrix:

and after applying the SVD, i got these:

notice that S (the first result) is a vector of (1 * 6), while it should be from size ( 6 * 6)

what's going on please?

Answer 1

The notation

$$S = \text{diag}(\lambda_1, \lambda_2, \ldots, \lambda_N)$$

defines the symmetric matrix

$$ S = \begin{bmatrix} \lambda_1 & 0 & \ldots & 0\\ 0 & \lambda_2 & \ldots & 0\\ \vdots & \vdots & \vdots & \vdots\\ 0 & 0 & \ldots & \lambda_N\\ \end{bmatrix} $$

Other notation than can helps to understand the situation is

$$d = \text{diag}(S)$$

which defines a vector with the elements in the main diagonal of $S$

$$ d = \begin{bmatrix} \lambda_1\\ \lambda_2\\ \vdots\\ \lambda_N\\ \end{bmatrix} $$

EDIT: The svd function gives you a vector of length $\text{min}(n,m)$ with the singular values of the matrix. In your case $n=6$ and $m=5$, therefore the length of the vector is $5$. As you said, the matrix $S$ is $6\times 5$, therefore in this case $S$ would have zeros everywhere except at the first $5$ elements in the main diagonal. That is, the last row of $S$ is all zeros. Although the above notation is not being used strictly, it could help to understand why a vector and not a matrix, which was your original question.

EDIT2: I change my wording in my firs edit in response to Mike's comment. $\text{min}(n,m)$ is the length of the vector $p$ that is returned by the svd function, which is the number of singular values that an $n\times m$ matrix can have, and some of them can be zero. I guess that if the rank $r$ of the matrix is less than $\text{min}(n,m)$, the function would return $\text{min}(n,m)-r$ singular values equal to zero in the vector. I could test this maybe later.

EDIT3: Here an example where the rank $r$ of the matrix is $3$ but $\text{min}(4,5) = 4$.

$\hspace{3cm}$

The svd function returns a vector of lenght $4$, with $3$ non-zero singular values and one singular value equal to $0$.

Answer 2

Every matrix $$ \mathbf{A} \in \mathbb{C}^{m\times n}_{\rho} $$ has a singular value decomposition

$$ \begin{align} \mathbf{A} &= \mathbf{U} \, \Sigma \, \mathbf{V}^{*} \\ % &= % U \left[ \begin{array}{cc} \color{blue}{\mathbf{U}_{\mathcal{R}}} & \color{red}{\mathbf{U}_{\mathcal{N}}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{S}_{\rho\times \rho} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % V* \left[ \begin{array}{c} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{V}_{\mathcal{N}}}^{*} \end{array} \right] \\ % % &= % U \left[ \begin{array}{cc} \color{blue}{\mathbf{U}_{\mathcal{R}}} & \color{red}{\mathbf{U}_{\mathcal{N}}} \end{array} \right] % Sigma \left[ \begin{array}{cccc|cc} \sigma_{1} & 0 & \dots & & & \dots & 0 \\ 0 & \sigma_{2} \\ \vdots && \ddots \\ & & & \sigma_{\rho} \\\hline & & & & 0 & \\ \vdots &&&&&\ddots \\ 0 & & & & & & 0 \\ \end{array} \right] % V \left[ \begin{array}{c} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{V}_{\mathcal{N}}}^{*} \end{array} \right] \\ % & = % U \left[ \begin{array}{cccccccc} \color{blue}{u_{1}} & \dots & \color{blue}{u_{\rho}} & \color{red}{u_{\rho+1}} & \dots & \color{red}{u_{n}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % V \left[ \begin{array}{c} \color{blue}{v_{1}^{*}} \\ \vdots \\ \color{blue}{v_{\rho}^{*}} \\ \color{red}{v_{\rho+1}^{*}} \\ \vdots \\ \color{red}{v_{n}^{*}} \end{array} \right] % \end{align} $$

The $S$ in your output is the list of singular values $\sigma$. The are the diagonal elements of the invertible matrix $\mathbf{S}$.

The $\rho$ singular values are ordered and satisfy $$ \sigma_{1} \ge \sigma_{2} \ge \dots \ sigma_{1} > 0/ $$

---

---

The target matrix has rank $\rho = 5$ as evidenced by the reduction. $$ \begin{align} \left[ \begin{array}{c|c} \mathbf{A} & \mathbf{I}_{6} \end{array} \right] \qquad &\mapsto \qquad \left[ \begin{array}{c|c} \mathbf{E_{A}} & \mathbf{R} \end{array} \right] \\ % \left[ \begin{array}{ccccc|cccccc} 5 & 2 & 1 & 1 & 4 & 1 & 0 & 0 & 0 & 0 & 0 \\ 0 & 1 & 1 & 0 & 3 & 0 & 1 & 0 & 0 & 0 & 0 \\ 3 & 4 & 1 & 0 & 2 & 0 & 0 & 1 & 0 & 0 & 0 \\ 4 & 0 & 0 & 0 & 3 & 0 & 0 & 0 & 1 & 0 & 0 \\ 3 & 0 & 2 & 5 & 4 & 0 & 0 & 0 & 0 & 1 & 0 \\ 2 & 5 & 1 & 5 & 1 & 0 & 0 & 0 & 0 & 0 & 1 \\ \end{array} \right] % &\mapsto % \left[ \begin{array}{ccccc|rrrrrr} \boxed{1} & 0 & 0 & 0 & 0 & \frac{45}{23} & -\frac{6}{23} & -\frac{21}{23} & -\frac{28}{23} & -\frac{9}{23} & 0 \\ 0 & \boxed{1} & 0 & 0 & 0 & -\frac{65}{23} & \frac{1}{23} & \frac{38}{23} & \frac{43}{23} & \frac{13}{23} & 0 \\ 0 & 0 & \boxed{1} & 0 & 0 & \frac{245}{23} & -\frac{2}{23} & -\frac{122}{23} & -\frac{178}{23} & -\frac{49}{23} & 0 \\ 0 & 0 & 0 & \boxed{1} & 0 & -\frac{77}{23} & -\frac{2}{23} & \frac{39}{23} & \frac{52}{23} & \frac{20}{23} & 0 \\ 0 & 0 & 0 & 0 & \boxed{1} & -\frac{60}{23} & \frac{8}{23} & \frac{28}{23} & \frac{45}{23} & \frac{12}{23} & 0 \\\hline 0 & 0 & 0 & 0 & 0 & \frac{435}{23} & \frac{11}{23} & -\frac{249}{23} & -\frac{286}{23} & -\frac{110}{23} & 1 \\ \end{array} \right] % \end{align} $$

---

The SVD for this matrix is

Column space $$ \mathbf{U} = \left[ \begin{array}{llllll} % r 1 \color{blue}{\phantom{-}0.509682} & \color{blue}{\phantom{-}0.374719} & \color{blue}{-0.163866} & \color{blue}{-0.142365} & \color{blue}{-0.0737315} & \color{red}{\phantom{-}0.739766} \\ % r 2 \color{blue}{\phantom{-}0.173038} & \color{blue}{\phantom{-}0.137867} & \color{blue}{-0.00979447} & \color{blue}{\phantom{-}0.911177} & \color{blue}{\phantom{-}0.346932} & \color{red}{\phantom{-}0.0187067} \\ % r 3 \color{blue}{\phantom{-}0.362913} & \color{blue}{\phantom{-}0.0169967} & \color{blue}{-0.619884} & \color{blue}{\phantom{-}0.134304} & \color{blue}{-0.535164} & \color{red}{-0.423452} \\ % r 4 \color{blue}{\phantom{-}0.309388} & \color{blue}{\phantom{-}0.514438} & \color{blue}{-0.0113491} & \color{blue}{-0.332591} & \color{blue}{\phantom{-}0.540673} & \color{red}{-0.486375} \\ % r 5 \color{blue}{\phantom{-}0.506436} & \color{blue}{-0.0388134} & \color{blue}{\phantom{-}0.753794} & \color{blue}{\phantom{-}0.0615524} & \color{blue}{-0.367462} & \color{red}{-0.187067} \\ % r 6 \color{blue}{\phantom{-}0.475792} & \color{blue}{-0.757718} & \color{blue}{-0.143042} & \color{blue}{-0.130562} & \color{blue}{\phantom{-}0.400559} & \color{red}{\phantom{-}0.0391141} \\ % \end{array} \right] $$

Singular values $$ \Sigma = \left[ \begin{array}{c} \mathbf{S} & \mathbf{0} \end{array} \right] = \left[ \begin{array}{ccccc} 12.6517 & . & 0 & 0 & 0 \\ 0 & 5.77087 & 0 & 0 & 0 \\ 0 & 0 & 4.74869 & 0 & 0 \\ 0 & 0 & 0 & 2.57851 & 0 \\ 0 & 0 & 0 & 0 & 0.657751 \\\hline 0 & 0 & 0 & 0 & 0 \\ \end{array} \right] $$

Row space $$ \mathbf{V} = \left[ \begin{array}{lllll} \color{blue}{0.580601} & \color{blue}{\phantom{-}0.407297} & \color{blue}{-0.157745} & \color{blue}{-0.6654} & \color{blue}{-0.17138} \\ % r 2 \color{blue}{0.397023} & \color{blue}{-0.490966} & \color{blue}{-0.743842} & \color{blue}{\phantom{-}0.198119} & \color{blue}{\phantom{-}0.0936634} \\ % r 3 \color{blue}{0.200313} & \color{blue}{-0.0529838} & \color{blue}{\phantom{-}0.120244} & \color{blue}{\phantom{-}0.347355} & \color{blue}{-0.906617} \\ % r 4 \color{blue}{0.428466} & \color{blue}{-0.625199} & \color{blue}{\phantom{-}0.608566} & \color{blue}{-0.189028} & \color{blue}{\phantom{-}0.139496} \\ % r 5 \color{blue}{0.530629} & \color{blue}{\phantom{-}0.446521} & \color{blue}{\phantom{-}0.192363} & \color{blue}{\phantom{-}0.601336} & \color{blue}{\phantom{-}0.347049} \\ \end{array} \right] $$