Approximate matrix by a rank 2 matrix using singular values

Question

I only understand the singular value decomposition process. Do I apply it to this matrix?

\begin{bmatrix} 0 & 0 & \pi \\ 0 & e & 0 \\ 1&0&0 \end{bmatrix}

What is the idea behind matrix approximation using this process?

Applying SVD, we have $\begin{bmatrix} 0 & 0 & \pi \\ 0 & e & 0 \\ 1&0&0 \end{bmatrix}^T\begin{bmatrix} 0 & 0 & \pi \\ 0 & e & 0 \\ 1&0&0 \end{bmatrix}= \begin{bmatrix} 1 & 0 & 0 \\ 0 & e^2 & 0 \\ 0&0&\pi^2 \end{bmatrix}$

How to proceed?

Answer 1

For a matrix $A \in \mathbb{R}^{m\times n}$, the best rank-$r$ approximation $R$ to $A$ (in the Frobenius norm sense, $r \leq rank(A)$) is given by

$$ R = \sum_{i=1}^r \sigma_i u_i v_i^T $$

where $\sigma_i$ is the $i$th largest singular value of $A$, $u_i$ is the $i$th column of the $U$ matrix, $v_i^T$ is the $i$th row of $V^T$ matrix in the singular value decomposition $A=U\Sigma V^T$.

Answer 2

The idea is as follows: we find the SVD of this matrix, which has the form $$ A = U \Sigma V^T = \pmatrix{u_1 & u_2 & u_3} \pmatrix{\sigma_1 \\&\sigma_2\\&& \sigma _3} \pmatrix{v_1^T\\v_2^T \\ v_3^T} = \sum_{i=1}^3\sigma_i u_i v_i^T $$ The rank-2 approximation is $$ A' = \pmatrix{u_1 & u_2 & u_3} \pmatrix{\sigma_1 \\&\sigma_2\\&& 0} \pmatrix{v_1^T\\v_2^T \\ v_3^T} = \sum_{i=1}^2\sigma_i u_i v_i^T $$

Answer 3

the matrix $A = \pmatrix{0 & 0 & \pi \cr 0 & e & 0\cr 1 & 0 & 0} = UDV^T = \pmatrix{1 & 0 & 0\\0&1&0\\0&0&1}\pmatrix{\pi & 0 & 0\cr 0 & e &0\cr 0 & 0& 1}\pmatrix{0 & 0&1\\0&1&0\\1&0&0}^T = \pi e_1 e_3^T + e \ e_2 e_2^T + e_3 e_1^T$

where $e_1, e_2$ and $e_3$ is the standard basis. so the rank $2$ approximation of $A$ is $$\pi e_1 e_3^T + e \ e_2 e_2^T.$$

Answer 4

Decomposition

By agreement, the SVD orders the singular values. The decomposition here is $$ \mathbf{A} = % \left( \begin{array}{ccc} 0 & 0 & \pi \\ 0 & e & 0 \\ 1 & 0 & 0 \\ \end{array} \right) % = \mathbf{U} \, \Sigma \, \mathbf{V}^{*} = % \left( \begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right) % \left( \begin{array}{ccc} \pi & 0 & 0 \\ 0 & e & 0 \\ 0 & 0 & 1 \\ \end{array} \right) % \left( \begin{array}{ccc} 0 & 0 & 1 \\ 0 & 1 & 0 \\ 1 & 0 & 0 \\ \end{array} \right). % $$

The approximation through order $k$ uses the first $k$ singular values. In your case, $k=2$ and the approximation is $$ \mathbf{A}_{2} = % \left( \begin{array}{ccc} 0 & 0 & \pi \\ 0 & e & 0 \\ 0 & 0 & 0 \\ \end{array} \right) % = \mathbf{U} \, \Sigma \, \mathbf{V}^{*} = % \left( \begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right) % \left( \begin{array}{ccc} \pi & 0 & 0 \\ 0 & e & 0 \\ 0 & 0 & \color{red}{0} \\ \end{array} \right) % \left( \begin{array}{ccc} 0 & 0 & 1 \\ 0 & 1 & 0 \\ 1 & 0 & 0 \\ \end{array} \right). % $$ The red $0$ shows that you have discarded $\sigma_{3}$.