Proximal Gradient of absolute value of linear function

139 Views Asked by At

I am working through

https://www.ipol.im/pub/art/2013/26/article.pdf

This is an attempt at simplyfying deriving (13) from (9).

One of the posed problems can be written as,

\begin{equation} \min_{\textbf{v}} \frac{1}{2}||\textbf{v} - \textbf{u}||_2^2 + |\textbf{y}\cdot (\textbf{v} - \textbf{x}) + c| \end{equation}

Where $\textbf{y}$, $\textbf{x}$, and $c$ are fixed.

Taking the gradient, I get the set of equations:

\begin{equation} 0 = \begin{cases} \textbf{v} - \textbf{u} - \textbf{y} & \rho(\textbf{v}) < 0\\ [\textbf{v} - \textbf{u} - \textbf{y}, \textbf{v} - \textbf{u} + \textbf{y}] & \rho(\textbf{v}) = 0\\ \textbf{v} - \textbf{u} + \textbf{y} & \rho(\textbf{v}) > 0 \end{cases} \end{equation} where $$\rho(\textbf{v}) := |\textbf{y}\cdot (\textbf{v} - \textbf{x}) + c|$$

this is where I get stuck.

I arrive at this inequality

\begin{equation} \min(\textbf{v} - \textbf{u} - \textbf{y}, \textbf{v} - \textbf{u} + \textbf{y}) \le 0 \le \max(\textbf{v} - \textbf{u} - \textbf{y}, \textbf{v} - \textbf{u} + \textbf{y}) \end{equation}

where min and max are done element wise, e.g.

\begin{equation} \min(v_i - u_i - y_i, v_i - u_i + y_i) \le 0 \le \max(v_i - u_i - y_i, v_i - u_i + y_i) \end{equation}

Intuitively, $\text{v} = \textbf{u} + \lambda \textbf{y}$, where $\lambda \in [-1, 1]$. However, I am getting stuck working with the constraint $\rho(\textbf{v}) = 0$ which leads to

$$\lambda = -\rho(\textbf{v})\frac{\textbf{y}}{||\textbf{y}||_2^2} $$

1

There are 1 best solutions below

0
On

the initial problem with the absolute value can be reformulated into a smooth constrained problem:

\begin{equation} \begin{aligned} \min_{\textbf{v}} & \frac{1}{2}||\textbf{v} - \textbf{u}||_2^2 + p + n \\ s.t. & \quad \textbf{y}\cdot (\textbf{v} - \textbf{x}) + c = p - n \\ & \quad p, n \ge 0 \end{aligned} \end{equation} where $p$ and $n$ capture the positive and negative part of $\textbf{y}\cdot (\textbf{v} - \textbf{x}) + c$, respectively. You can then tackle this problem using constrained optimization methods.