Solving Matrix Equation using SVD

39 Views Asked by At

I'm reading this paper by Bishop and Tipping. They solve the equation $$(SC^{-1} - I)W = 0$$ Where $W \in \mathbb{R}^{d \times q}$ and $S , C \in \mathbb{R}^{d \times d}$ and $C = WW^T + \sigma^2 I$ and $q < d$. $S$ is the sample Covariance matrix (Covariance of $t \in \mathbb{R}^d$). It's assumed that the lowest $(d - q)$ eigenvalues of $S$ are constant (it's not very well clarified if this is assumed or not).

On page $619$. They have given the solutions. There are $3$ Cases.
Case 1: $W = 0$
Case 2: $C = S$
Case 3: Otherwise
I'm stuck the the solutions of Case 3 For Case3, They use the SVD of $W$ (eqn $(12)$) $$W = ULV^T$$ Where $U \in \mathbb{R}^{d \times q}$, $L \in \mathbb{R}^{q \times q}$ the the diagonal matrix of singular values and $V \in \mathbb{R}^{q \times q}$ is an orthogonal matrix.
I was using these properties to simplify, $$VV^T = V^TV = I_q$$ $$U^TU = I_q$$ But haven't gone any far in obtaining eqn $(13)$. It looks like they are explicitly using the information, $W \neq 0$ and $S \neq C$ for this case (since this is case 3) but I don't know how. $$SUL = U(\sigma^2 I + L^2)L$$ Any help on how to go from eqn $(12)$ to $(13)$ Will be highly appreciated.