Minimization of a summation and a norm

56 Views Asked by At

I have been trying to figure out how to minimize

$$\|y-x\|_2^2 +\mu\sum_{i=1}^{n-1} (y_{i+1}-y_i)^2$$

where $y$ is a vector and $x$ is a noisy data set vector by putting it into the form of

$$\|Ay-b\|_2^2$$

and minimizing from there. I understand how to deal with the left half of the equation but the summation is what's throwing me off.

2

There are 2 best solutions below

2
On BEST ANSWER

The objective function can be written as $$ f(y) = \| y - x\|_2^2 + \mu \| Dy \|_2^2 $$ where $$ D = \begin{bmatrix} -1 & 1 & 0 & 0 & \cdots & 0 & 0\\ 0 & -1 & 1 & 0 & \cdots & 0 & 0\\ \vdots & & \ddots & \ddots& & \vdots & \vdots \\ 0 & 0 & 0 & 0 & \cdots & -1 & 1 \end{bmatrix}. $$ The gradient of $f$ is $$ \nabla f(y) = 2(y - x) + 2\mu D^T Dy. $$ To minimize $f$, you can set the gradient equal to $0$ and solve for $y$.

0
On

$A$ is the shift operator (where $a_{i, i+1} = 0,$ and $a_{i,j}=0$ otherwise.)