Are all three matrices in Singular Value Decomposition orthornormal?

Question

For Singular Value Decomposition (SVD)

M = USV'

I know that U and V are orthogonal matrices, but does S the diagonal matrix also have to be orthogonal?

I know that not all diagonal matrices are orthogonal.

Answer 1

Nope, $$\begin{bmatrix} 2 & 0 \\ 0 & 1\end{bmatrix}= \begin{bmatrix} 1 & 0 \\ 0 & 1\end{bmatrix}\begin{bmatrix} 2 & 0 \\ 0 & 1\end{bmatrix}\begin{bmatrix} 1 & 0 \\ 0 & 1\end{bmatrix}$$

is a valid SVD decomposition.

$$\begin{bmatrix} 2 & 0 \\ 0 & 0\end{bmatrix}= \begin{bmatrix} 1 & 0 \\ 0 & 1\end{bmatrix}\begin{bmatrix} 2 & 0 \\ 0 & 0\end{bmatrix}\begin{bmatrix} 1 & 0 \\ 0 & 1\end{bmatrix}$$

is a valid SVD decomposition too.

Remark: from your phrasing of using the word "orthogonal orthonormal", I have a feeling that you might interpreted an orthgonal matrix as a square matrix of which its columns are orthogonal, an orthogonal matrix is a matrix $Q$ where $Q^TQ=I$ and $QQ^T=I$.

If $S$ is an orthogonal matrix, then $M$ as product of orthogonal matrices will be orthogonal but SVD is not restricted to orthogonal matrices.

Answer 2

$S$ can be any diagonal matrix with non-negative entries (although they are usually written in descending order). Diagonal matrices are orthogonal if and only if the diagonal entries are all either $1$ or $-1$.

But more importantly, $S$ doesn't need to be square, and non-square matrices can never be orthogonal. In fact, $S$ has the same dimension as $M$.