How to prove the projection theorem?

585 Views Asked by At

For any point $y$, the projection onto a nonempty and closed convex set $X$ is defined as

$$\Pi_X(y) := \arg\min_{x \in X} \frac{1}{2} \| x - y \|_2^2.$$

I am trying to prove for any point $x\in X$,

$$\langle y - \Pi_X(y) , x - \Pi_X(y) \rangle \le 0$$

First, let $f(x)=\frac{1}{2}\|x-y\|_2^2$, $\nabla f(x)=x-y$. $f(x)$ is a convex function because $\nabla^2f(x)=1\ge0$. So,

$$f(x)\ge f(\Pi_X(y))+\nabla f(\Pi_X(y))^T(x-\Pi_X(y))=f(\Pi_X(y))+(\Pi_X(y)-y)^T(x-\Pi_X(y))\\ \Rightarrow (\Pi_X(y)-y)^T(x-\Pi_X(y)) = \langle \Pi_X(y)-y, x-\Pi_X(y) \rangle \le f(x) - f(\Pi_X(y))\\ \Rightarrow \langle y - \Pi_X(y) , x - \Pi_X(y) \rangle \ge f(\Pi_X(y)) - f(x)$$

And because $\Pi_X(y)$ is the projection of $y$ onto $X$, $f(\Pi_X(y)) \le f(x)$, $f(\Pi_X(y)) - f(x)\le0$. At this point, I can't continue my proof. Is there something wrong with my process?

1

There are 1 best solutions below

2
On BEST ANSWER

Let $z:=\Pi_X(y)$, let $x\in X$, $t>0$. Then $$ 0 \le f(z+t(x-z)) - f(z) = t(z-y, x-z) + \frac {t^2}2\|x-z\|^2. $$ This is the Taylor expansion of the quadratic function $f$. Now divide by $t$ and pass to the limit $t\searrow0$.