Proximal operators on Balls (Projection)

153 Views Asked by At

I was following Gabriel Peyre - Dictionary Learning for Denoising tutorial, In section 21 it is given Proximal operator over a ball $B_\epsilon$ of radius $\epsilon$ as $$\text{Proj}_{B_\epsilon(y)}(u) = y + (u-y) \max({1 , \frac{\epsilon}{||{u-y}||} })$$

I would like to know the proof of the above statement. Thanks in advance

1

There are 1 best solutions below

10
On BEST ANSWER

You are trying to solve $\min \{ f(x) | x \in C \}$, where $f(x) = {1\over 2} \|u-x\|^2$ and $C = \bar{B}(y, \epsilon)$. Since $C$ is compact, we know there is a solution $\hat{x}$. Since the Euclidean norm is strictly convex, we see that the solution is unique.

Since $f$ and $C$ are convex, we have that $\hat{x}$ solves the problem iff $D f(\hat{x}) (x-\hat{x}) \ge 0$ for all $x \in C$.

In this particular problem, this says that $\hat{x}$ solves the problem iff $\langle \hat{x}-u, x-\hat{x} \rangle \ge 0$ for all $x$ such that $\|x-y\| \le \epsilon$.

If $\|u-y\| \le \epsilon$, we have $\langle 0, x-\hat{x} \rangle = 0$ for all $x$, so we see that $\hat{x}=u$ is the solution.

If $\|u-y\| > \epsilon$, let $\hat{x} = {\epsilon \over \|u-y\|} (u-y)+y$, then \begin{eqnarray} \langle \hat{x}-u, x-\hat{x} \rangle &=& (1-{\epsilon \over \|u-y\|}) \langle y-u, x-\hat{x} \rangle \\ &=& (1-{\epsilon \over \|u-y\|}) \langle y-u, x-y+ {\epsilon \over \|u-y\|} (y-u)\rangle \\ &=& (1-{\epsilon \over \|u-y\|}) \left( \langle y-u, x-y \rangle + \langle y-u, {\epsilon \over \|u-y\|} (y-u)\rangle \right) \\ &=& (1-{\epsilon \over \|u-y\|}) \left( \langle y-u, x-y \rangle + \epsilon \|u-y\| \right) \\ &\ge& 0 \end{eqnarray} Where the inequality follows from Cauchy-Schwarz and $\|x-y\| \le \epsilon$.

Combining the two results gives the desired answer.