stuck following proof that the rank r SVD approximation of a matrix D, D_r minimizes the Froebenius norm D-D_r

Question

I am stuck following a proof of the Eckart-Young Theorem which states that the rank r approximation of a matrix $D_r$ minimizes the Frobenius norm to the original matrix $D$.

Let the SVD decomposition of a matrix $D$ be $L\Delta R^T$.

The Frobenius norm is defined as:

let $A = D-D_r$, then

$$||A||_F = \sqrt(\sum_i\sum_j A_{ij}^2)$$

from there it follows:

$$||A||_F = ||D-D_r||_F=||L\Delta R^T - D_r||_F = ||\Delta - L^TD_rR||_F$$

I do not see how in the previous line the last equality holds. Can someone point out what I am missing?

Answer 1

It's because the Frobenius norm is invariant under left or right multiplication of unitary matrices (it's easy to see this if you know that the Frobenius norm is induced by the inner product $\langle A,B\rangle=\operatorname{tr}(AB^T)$), and the $L,R$ in your singular value decomposition are unitary matrices.

Answer 2

The answer to your question is that the Frobenius norm is a unitarily invariant norm, which implies that it is invariant under unitary transformations i.e. $||QA||_F=||A||_F$ if the $Q$ matrix is a unitary matrix.

As in the svd decomposition, the matrices L and R are unitary, then every transformation of the norm using them remains the norm the same. That means that:

$||A||_F=||D-D_r||_F=||L\Delta R^T-D_r||=||L^TL\Delta R^TR-L^TD_rR||_F=||\Delta-L^TD_r R||_F$