Different ways of solving $\underset{\mathbf{s}}{\text{min}}\;\|F\mathbf{s}-\mathbf{x}\|_{l_2}^2 + \|W\mathbf{s}\|_{l_2}^2$ least square problem?

66 Views Asked by At

The problem that I am going to describe arises from compressed sensing technique and after using weighted least squares it can be transformed into the following least squares problem:

$$\underset{\mathbf{s}}{\text{min}}\;\|F\mathbf{s}-\mathbf{x}\|_{l_2}^2 + \|W\mathbf{s}\|_{l_2}^2$$

where $F\in\mathbb{C}^{M\times N}$, $\mathbf{s}\in \mathbb{R}^N$, $\mathbf{x}\in\mathbb{R}^M$ and $W\in \mathbb{R}^{N\times N}$ (diagonal matrix).

My approach to solve the problem was, let $\mathbf{e} = F\mathbf{s}-\mathbf{x}$ then $$ \frac{\partial}{\partial \mathbf{s}} \left[ \mathbf{e}^T\mathbf{e} + (W\mathbf{s})^TW\mathbf{s} \right] = -2F^H\mathbf{x} + 2F^HF\mathbf{s} + 2W^TW\mathbf{s} = 0 $$ where $F^H$ is the conjugate transpose. Then, the solution is: $$\mathbf{s} = (F^HF+W^2)^{-1}F^H\mathbf{x}.$$

Is this solution correct? In the notes that I have the author gives the follwoing solution: $$\mathbf{s} = W^{-1}F^H\Gamma$$ where $\Gamma$ is obtained by solving system of linear equations, $$(FW^{-1}F^H+E)\Gamma = \mathbf{x}$$ being $\mathbf{E}$ the unity matrix (I guess that is a matrix with all 1's but I am not sure).

Do you know how it gets this solution?

After that I know that the problem must be solved iteratively since $W$ values are not known but I do not have any problem about that.

Thanks!