Lower bound on sum of two least square problems

40 Views Asked by At

Let $D, E \in \mathbb{R}^{m \times n}$ be full rank matrices where $m \geq n$.


Question:

Is $ \Big( \min_{x \in \mathbb{R}^{n}} \|Dx-b\|^2+\|Ex-y\|^2\ \Big) >0 $


My try:

If we let the gradient of the objective function be zero we get the following: $$ D^{\top}Dx*+E^{\top}Ex*=D^{\top}b+E^{\top}y $$

Matrices $D$ and $E$ are full rank so $D^{\top}D$ and $E^{\top}E$ are positive definite matrices. Thus, $C= D^{\top}D+E^{\top}E$ is a positive definite matrix. As a result $x*$ is unique and can be found as follows: $$ x*=c^{-1}(D^{\top}b+E^{\top}y) $$


Question:

How can I rule out $x*$ does not make the objective value zero? Or conditions that rule it out?

1

There are 1 best solutions below

0
On

First, if $\min_{x \in \mathbb{R}^n} \lVert Dx-b\rVert^2>0$ or $\min_{x \in \mathbb{R}^n} \lVert Ex-y\rVert^2>0$, then the objective value is clearly nonzero. To check it, evaluate the functions at their minimum for each of these problems (minimum reached for $x'=(D^TD)^{-1}Db$ and $x''=(E^TE)^{-1}Ey$ respectively).

Now, assuming that $\min_{x \in \mathbb{R}^n} \lVert Dx-b\rVert^2=0$ and $\min_{x \in \mathbb{R}^n} \lVert Ex-y\rVert^2=0$, we would like to know when $\min_{x \in \mathbb{R}^n} \lVert Dx-b\rVert^2 + \lVert Ex-y\rVert^2=0.$

We have: $$ \begin{align} &\min_{x \in \mathbb{R}^n} \lVert Dx-b\rVert^2 + \lVert Ex-y\rVert^2=0\\ \iff &\lVert Dx^*-b\rVert^2 + \lVert Ex^*-y\rVert^2=0 \\ \iff & \lVert Dx^*-b\rVert^2 = 0 \text{ and } \lVert Ex^*-y\rVert^2=0\\ \iff &x^*=x' \text{ and } x^*=x''\\ \iff & x'=x'' \end{align} $$

So a necessary and sufficient condition for $\min_{x \in \mathbb{R}^n} \lVert Dx-b\rVert^2 + \lVert Ex-y\rVert^2$ to be zero is $\lVert Dx'-b\rVert^2=0$ and $\lVert Ex'-y\rVert^2=0$ for $x'= (D^TD)^{-1}Db.$