Minimize Function over Convex Subset

1.1k Views Asked by At

Suppose that C is a closed convex subset of $\mathbb R^n$ and $x \in \mathbb R^n$. The projection of $\mathbf x$ onto C is the closest point $\mathbf y \in C : \mathbf z = \mathbf y$ minimizes ||$\mathbf z - \mathbf x$||$_2$ over all $\mathbf z \in C$.

(a) Show that the minimum exists.

(b) Show that there is only one minimizer: that is, show that if $\mathbf y_1$ and $\mathbf y_2$ both minimize ||$\mathbf x - \mathbf z$|| then $\mathbf y_1 = \mathbf y_2$

My ideas:

For (a), to show that the minimum exists I will need to show that it is closed and bounded. We already know it's closed because C is a closed convex subset so I just need to show it's bounded. Am I wanting to minimize ||$\mathbf x - \mathbf z$||$_2$? This is just the Euclidean norm which would be $\sqrt{(\mathbf z - \mathbf x)^2}$=$\sqrt{\mathbf z^2 -\mathbf z \mathbf x -\mathbf x \mathbf z + \mathbf x^2}$. I'm not sure where to go from here...

For (b) I want to use proof by contradiction- so I will assume that $\mathbf y_1$ and $\mathbf y_2$ both minimize the function and then I'll somehow see that they are in fact equal. How do I set up this argument?

2

There are 2 best solutions below

2
On BEST ANSWER

For (a), the set $C$ may not be bounded so you need to do a little more work. I make the additional assumption that $C$ is nonempty. Let $z_0\in C$. If $\|x-z\|_2<\|x-z_0\|$ then $z\in \overline{B_{\|x-z_0\|}(z_0)}\cap C$, which is closed and bounded. Hence $\|z-x\|$ has a minimum in $\overline{B_{\|x-z_0\|}(z_0)}\cap C$, which must be the minimum in $C$.

For (b), suppose $\|x-z_1\|$ and $\|x-z_2\|$ are minima, and consider the line $f(t)=tz_2+(1-t)z_1$ which is contained in $C$ by convexity. But $\|x-f(t)\|^2$ is a nonnegative quadratic, so it has a unique minimum which must be between $0$ and $1$ since $\|x-f(0)\|^2=\|x-f(1)\|^2$. But this contradicts the minimality of $\|x-z_1\|$ and $\|x-z_2\|$.

1
On

For (a), because the objective function is continuos and the constraints set is compact, therefore, there always exists a solution (Existence). For (b), you want to prove uniqueness, you should prove the objective function is strictly convex. In your case, because l2 norm is strictly convex, so we have uniqueness.

BTW, if C is a subspace, then based on classical projection theorem, existence and uniqueness are always guarantee.