I've been studying some articles on the Generalized Alternate Projection (GAP) algorithm recently, but have a little doubt about the derivation of this algorithm.
In paper Generalized Alternating Projection for Weighted-ℓ2,1 Minimization with Applications to Model-based Compressive Sensing they said
Given $\theta$, the update of w is simply an Euclidean projection of $\theta$ on the linear manifold.
($\Phi \in \mathbb{R} ^{r \times n} ,r<n,$ and $\Phi \Phi^{T}$ is invertible.)
What puzzles me is the format of its Euclidean projection: why is it $\Phi^{T}(\Phi \Phi^{T})^{-1}(y-\Phi \theta)$ (right inverse of $\Phi$ * $(y-\Phi \theta)$) instead of $(\Phi^{T} \Phi)^{-1}\Phi^{T} (y-\Phi \theta)$ (left inverse of$\Phi$ * $(y-\Phi \theta)$)?
The iterative format of GAP that I have seen in other papers is the same, so the possibility of a clerical error by the author can basically be ruled out.
Or should I be asking how to get the following iterative format by a Euclidean projection of $\theta$ on the linear manifold $y=\Phi w$: $$w^{(t)}=\theta^{(t-1)}+\Phi^{T}(\Phi \Phi^{T})^{-1}(y-\Phi \theta^{(t-1)})$$
I tried to give an explanation, if there is a mistake, please do not hesitate to enlighten.
According to (2.3) since $\lambda ^{(t)} \left \| \theta \right \| _{\ell^{ \mathcal{G} ,\beta}_{2,1} }$ can be seen as a constant term, it can be ignored,then the (2.3) could be simplified:
$$\min f(w)=\frac{1}{2} \left \| w-\theta^{(t)} \right \| ^{2}\ \mathbf{s.t.} \ y=\Phi w$$
its Lagrange function(where $\lambda$ is Lagrange multiplier)： $$\mathcal{L} _{f}(w,\lambda)=\frac{1}{2} \left \| w-\theta^{(t)} \right \| ^{2}+\lambda^{T}(y-\Phi w)$$ So there is $\nabla\mathcal{L} _{f}=0_{((n+r)\times 1)} $ i.e. $$ \begin{cases} \frac{\partial \mathcal{L} _{f}}{\partial w} = w-\theta^{(k)}-\Phi^{T} \lambda =0_{(n \times 1)}\\ \frac{\partial \mathcal{L} _{f}}{\partial \lambda} =y-\Phi w =0_{(r \times 1)} \end{cases} $$ then $$w=\theta^{(k)}+\Phi^{T}\lambda$$ and we try to prove that $$w=\theta^{(k)}+\Phi^{T}(\Phi \Phi^{T})^{-1}(y-\Phi \theta^{(k)})$$and we have $$\begin{align*} w-\theta^{(k)}&=\Phi^{T}\lambda \\ \Phi(w-\theta^{(k)})&= \Phi \Phi^{T}\lambda \\ (\Phi \Phi^{T})^{-1}\Phi(w-\theta^{(k)})&= \lambda \\ (\Phi \Phi^{T})^{-1}(y-\Phi \theta^{(k)})&=\lambda \\ \Phi^{T}(\Phi \Phi^{T})^{-1}(y-\Phi \theta^{(k)})&=\Phi^{T}\lambda \\ \Phi^{T}(\Phi \Phi^{T})^{-1}(y-\Phi \theta^{(k)})&=w-\theta^{(k)} \\ w&=\theta^{(k)}+\Phi^{T}(\Phi \Phi^{T})^{-1}(y-\Phi \theta^{(k)}) \end{align*}$$ The above equation is the required equation.