Orthogonal Projection onto the $ {L}_{2} $ Unit Ball

Question

On an article I'm reading, I find that: if $v$ is a vector, the projection of of $v$ on the unit ball is: $$p(v)=\frac{v}{\max\{1,\|v\|\}}$$ I know that a projection of a point $v$ into a space is the nearest point to $v$ inside the space..why the expression above?

Answer 1

I presume the setting is a Hilbert space $\mathbb{H}$.

Then the projection onto the closed unit ball $\bar{B}$ is given by a solution to $\min_{ x \in \bar{B}} \|x-v\|$.

If $v=0$, it is clear that the solution is $x=0$, so we will assume $v \neq 0$ in the sequel.

We can write any $x \in \mathbb{H}$ as $x = \lambda v + w$, where $w \bot v$. In particular, we have $\|x\|^2 = \lambda^2 \|v\|^2 + \|w\|^2$, and so $\|x-v\|^2 = (1-\lambda)^2 \|v\|^2 + \|w\|^2$. Hence if $x = \lambda v + w \in \bar{B}$, we see that $\lambda v \in \bar{B}$, and $\|\lambda v-v\|^2 \le \|x-v\|^2$.

If we let $V = \operatorname{sp} \{v\}$, we see that $\min_{ x \in \bar{B}} \|x-v\| = \min_{ \lambda v \in \bar{B}} (1-\lambda)^2 \|v\|^2 $, which is a one dimensional problem.

Since $\lambda v \in \bar{B}$ iff $|\lambda| \le {1 \over \|v\|}$, we see that the problem is solved by $\lambda = \min(1,{1 \over \|v\|} )$, that is, $x= \min(1,{1 \over \|v\|} )v$.

It is straightforward to see that this is the same as $p(v) $ above.

Answer 2

You can do it using KKT Conditions.

Here is something I wrote once doing so (Solution to Home Work exercise I had):

I wrote MATLAB code which implements them both at Mathematics StackExchange Question 2338491 - GitHub.
There is a test which compares the result to a reference calculated by CVX.

Answer 3

An alternative solution that works in any inner product space: if $v$ is already in the unit ball, then clearly $p(v)=v$. We can thus assume that $\lVert v\rVert>1$. Given an arbitrary $a$ in the unit ball, it suffices to show that

$$\lVert v-a\rVert^2 \geq \left\lVert v-\frac v{\lVert v\rVert}\right\rVert^2.$$

Note that the RHS rewrites as $(\lVert v\rVert-1)^2$, and expanding the squares we find the equivalent inequality:

$$\lVert v \rVert^2 - 2\langle v,a \rangle + \lVert a \rVert^2 \geq \lVert v\rVert^2 - 2\lVert v\rVert +1, $$ which simplifies as $$ \lVert a \rVert^2 - 2\langle v,a \rangle + 2\lVert v \rVert -1 \geq 0.$$

To prove this last inequality, note that $$\begin{align} \lVert a \rVert^2 - 2\langle v,a \rangle + 2\lVert v \rVert -1 &\geq \lVert a \rVert^2 - 2\lVert v \rVert\lVert a \rVert + 2\lVert v \rVert -1 \tag{1} \\ &= \lVert a \rVert^2 + 2\lVert v \rVert(1-\lVert a \rVert) -1 \\&\geq \lVert a \rVert^2 + 2(1-\lVert a \rVert) -1 \tag{2} \\&= (\lVert a \rVert-1)^2 \geq 0.\end{align}$$

$(1)$: Cauchy-Schwarz inequality
$(2)$: multiply both sides of $\lVert v \rVert \geq 1$ by the nonnegative scalar $1-\lVert a \rVert$.