Minimum Error Solutions to Restricted Linear Systems

32 Views Asked by At

Suppose I have a linear system $Ax=b$ where $A_{n \times n}$ is square and full-rank. I want to restrict the space of solutions by a matrix $P_{n \times m}$ where $m << n$ and P has column rank $m$ (that is, $P^{\dagger} P = Id$). That means, I want to minimize: $$ \left|APx'-b\right|^2 $$ Normally, this is a standard problem. However, consider the residue $r = APx'-b$ and the associated error $e = x-Px'$, where this means that $r=Ae$. I would like to get a solution that minimizes $\left|e\right|^2$ rather than the usual $\left|r\right|^2$. If I am not mistaken, this is equivalent to getting a solution where $e \in Null(P^T)$ (intuitively: the error is orthogonal to the columns of $P$, or alternatively, the solution $x'$ is the optimal solution spanned by $P$ in terms of error). One could just pose this as a linearly-constrained-quadratic-minimization as follows: $$ (x',e) = argmin\left|APx'-b\right|^2,s.t.\ APx'-b = AWe, $$ where $W$ is a basis (in its columns) for $Null(P^T)$ (so $P^TW=0$). Note that $A$ is invertible, so the condition is equivalent to what we described above. One could use the typical Lagrangian approach for this, but it creates a huge system. Is there a nicer closed-form way to solve this? Note that inverting $A$ should be avoided as $n$ could be very large. However, suppose that $m$ is not so big, and thus (pseudo-)inverting $P$ is not a big deal.