How to show $x_* \in R(A^T) +z $ is the global minimizer of $\min \frac{1}{2}\|x-z\|_2^2$ over $Ax=b$?

234 Views Asked by At

Consider the following problem:

$$ \min_{Ax=b} \frac{1}{2}\|x-z\|_2^2 $$ where $z \in \mathbb{R}^{n}$ is a given vector, $x \in \mathbb{R}^{n}$, $A \in \mathbb{R}^{m \times n}$ and $b \in \mathbb{R}^{m}$.

We know that the problem is a convex problem, because the constraint is affine and the objective function is convex. Also, the objective function is strongly convex so the problem has a unique global minimizer. If we assume that $x_*$ is the global minimizer then using Variational Inequality we can show that $x_* \in R(A^T) +z $.

My question is the reverse:

Show that for $x_* \in S$, $x_* \in R(A^T) +z $ is the global minimizer of $\min \frac{1}{2}\|x-z\|_2^2$ over $Ax=b$?

1

There are 1 best solutions below

0
On BEST ANSWER

$$ x_* -z \in R(A^T)=\{N(A)\}^{\perp} $$

Therefore, every vector in null space of $A$ is perpendicular to $x_*-z$. So,

$$ \langle x_*-z, y \rangle = 0 \,\,\, \forall y \in N(A) \tag{1} $$

Since $x_* \in S$, $Ax_*=b$ and $Ax=b\,\,\, \forall x \in S$. Therefore,

$$ N(A)= \{x-x_* \mid x \in S\} $$

So from (1) $\langle x_*-z, x_* -x \rangle = 0 \,\,\, \forall x \in S$.

Let $y=x_*-z$ and $w=x_* -x$ so $\langle y,w \rangle=0$, Pythagorean for $y,w$ yields

$$ \|y-w\|^2=\|y\|^2+\|w\|^2 \rightarrow \|x-z\|^2=\|x_*-z\|^2+\|x_* -x\|^2 \,\,\forall x \in S $$

Therefor,

$$ \|x_*-z\|^2 \leq \|x-z\|^2 \,\,\forall x \in S $$

Hence, $x_*$ is the global minimizer.