Show that there is a unique solution to $\min \left\lVert Ax-b \right\rVert_2$

70 Views Asked by At

Let $A \in \mathbb{R}^{m \times n}$ have rank $n$ and let $b \in \mathbb{R}^m$. Show that there is a unique solution to $$\min h(x)$$ for $h(x) := \left\lVert Ax-b \right\rVert_2$ on $M := \{ x \in \mathbb{R}^m \mid x \ge 0 \}$ by examining the equivalent problem $\min h^2(x)$ on $M$.

I do not see how to do this. The problem is the fact that $M$ restricts the signs of the entries of $x$. I guess we have to use that $Ax = b$ has a unique solution (not necessarily in $M$) somehow, but I do not see how.

Could you help me?

1

There are 1 best solutions below

3
On BEST ANSWER

The first step to proving this is to notice that $h^2$ is strictly convex. This follows since $\frac{d^2}{dx_idx_j}(h^2)=2(A^tA)_{ij}$, and since $A$ has rank $n$, we have that $A^tA$ has rank $n$, and thus has no kernel. Since $(A^tAv,v)=(Av,Av)$ we see that $A^tA$ is strictly positive definite.

Now, on a convex domain, a strictly convex function has a unique minimum. This follows from the fact that if we have two minima, $x_1,x_2$, we have that $$h^2(\frac{1}{2}(x_1+x_2))< \frac{1}{2}h^2(x_1)+\frac{1}{2}h^2(x_2)$$ Which contridicts the assumption that $x_1,x_2$ are minima.