From the paper for Generalized low-rank models by Stephen Boyd, this Frobenius loss function has been used using SVD. Can someone explain it to me the following equation? Is U inverse is equal to U transpose here or what has been done here as A = U sigma V transpose? And if we make it like sigma = U inverse A V inverse or what's going on. also why it is equal to sigma - U trans XY V? Please guide me. Thanks
There is a theorem called the Eckart-Young-Mirsky Theorem which states the following.
$$A = U \Sigma V^{T} $$ $$ A_{k} = \sum_{i=1}^{k} \sigma_{i} u_{i} v_{i}^{T} $$ $$ \| A - A_{k} \|_{2} = \| \sum_{i=k+1}^{n} \sigma_{i} u_{i} v_{i}^{T} \| = \sigma_{k+1} $$
Note
$$ \| A - A_{k} \|_{F}^{2} = \| \sum_{i=k+1}^{n} \sigma_{i} u_{i} v_{i}^{T} \|_{F}^{2} = \sum_{i=k+1}^{n} \sigma_{i}^{2} $$
Now...utilizing your stuff
$$ \| A - XY \|_{F}^{2} = \| \Sigma - U^{T} XY V \|_{F}^{2} $$ From above we have seem that because of the unitary invariance under the 2 norm that $U,V$ go away and we're left with simply the singular value matrix when approximating $A$ so that is why we have $\Sigma$ there.
If you read it define $X,Y$ neatly within the Eckart Mirsky theorem to be a low-rank approximation. We end up with
$$X = U_{k}\Sigma_{k}^{\frac{1}{2}} Y = \Sigma_{k}^{\frac{1}{2}} V_{k}^{T} $$ $$ XY = A_{k} $$ $$ \| \Sigma - U^{T} A_{k} V\|_{F}^{2} $$ But those are unitary..we'll end up with $$\| \Sigma - \Sigma_{k} \|_{F}^{2} $$