Asymmetric Least Squares Conversion from Equation to Matrix

214 Views Asked by At

In solving for asymmetric least squares baseline correction as defined in the article by Eilers and Boelens, the general equation is defined as:

$$S = \displaystyle\sum_i w_i (y_i-z_i)^2 + \lambda \displaystyle\sum_i (\Delta^2 z_i)^2$$

The paper then immediately states "The minimization problems leads to the following system of equations:'"

$$(W + \lambda D' D)z = Wy$$

where $W = \operatorname{diag} (w)$ and $Dz = \Delta^2 z$.

The problem is that I really can't figure out this transition. I understand that we set $S=0$, but the only way I see this solving the equation is if we modify the general equation to be:

$$S = \displaystyle\sum_i w_i (y_i-z_i) + \lambda \displaystyle\sum_i (\Delta^2)^2 z_i$$

There must be some way of clearing out the variables, but I can't figure it out. (Thanks in advance for your help!)

1

There are 1 best solutions below

0
On BEST ANSWER

If $W = \operatorname{diag} (w)$ is nonnegative, then the quadratic cost function can be written in the form

$$\begin{array}{rl} S &= \|W^{\frac{1}{2}} (y - z)\|_2^2 + \lambda \|D z\|_2^2\\\\ &= (y-z)^T W (y-z) + \lambda \, z^T D^T D z\\\\ &= \begin{bmatrix} y\\ z\end{bmatrix}^T \begin{bmatrix} W & -W\\ -W & W\end{bmatrix} \begin{bmatrix} y\\ z\end{bmatrix} + \lambda \begin{bmatrix} y\\ z\end{bmatrix}^T\begin{bmatrix} O & O\\ O & D^T D\end{bmatrix} \begin{bmatrix} y\\ z\end{bmatrix}\\\\ &= \begin{bmatrix} y\\ z\end{bmatrix}^T \begin{bmatrix} W & -W\\ -W & W + \lambda \, D^T D\end{bmatrix} \begin{bmatrix} y\\ z\end{bmatrix}\end{array}$$

Taking the gradient of $S$ with respect to $(y,z)$, we obtain

$$\nabla S = 2\begin{bmatrix} W & -W\\ -W & W + \lambda \, D^T D\end{bmatrix} \begin{bmatrix} y\\ z\end{bmatrix}$$

As the gradient must vanish at the minimum, we obtain the following linear system

$$\begin{bmatrix} W & -W\\ -W & W + \lambda \, D^T D\end{bmatrix} \begin{bmatrix} y\\ z\end{bmatrix} = \begin{bmatrix} 0\\ 0\end{bmatrix}$$

which yields the equation

$$(W + \lambda \, D^T D) z = W y$$

and also the equation $W y = W z$. Hence, $\lambda \, D^T D z = 0$.