I was hoping you guys could help me out with a linear algebra problem which is stumping me (not homework).
Lets say I have a $n_1 \times m$ matrix $A$ such that $n_1 < m$ and a $n_1$-dimensional target vector $b_1$. If I solve $Ax = b_1$, this is under-determined and I can find lots of solutions.
Now consider I have a $n_2 x m$ matrix $C$, where now $n_2 > m$, and the first $n_1$ rows of $C$ is the matrix $A$ and the last $n_2-n_1$ rows of $C$ is some other matrix $B$. I have another $n_2$-dimensional target vector $b_2$. The first $n_1$ elements of $b_2$ is identical to $b_1$. If I try to solve $C x = b_2$, this is over-determined and I can't find any solutions. I can find the pseudo-inverse of $C$ to give me the vector $x$ which is away from a solution in a least-squares way.
My question is, how do I find a pseudo-solution of the over-determined equation with the additional constraint that the under-determined part of the problem defined by $Ax = b_1$ is still satisfied. That is, I want the first $n_1$ rows to be exactly satisfied in my pseudo solution, and the final $n_2 - n_1$ rows to be least squares in that restricted space. Of course I could assume some solution to the under-determined problem and then work in that space explicitly, but I want to use the freedom within the under-determined solutions to aid in the minimization of the error in the rest of the space.
Please let me know if any of this confuses you. Your help is much appreciated.
You can parameterize the solutions of $Ax=b_1$ as $$ x = A^{+}b_1+(I-A^{+}A)w $$ and then minimize $\|Bx-b_3\|_2$ where $b_3$ consists of the last $n_2-n_1$ elements of $b_2$. \begin{eqnarray*} && \|Bx-b_3\|_2 \\ & =& \|B(A^{+}b_1+(I-A^{+}A)w)-b_3\|_2 \\ & =& \|(B-BA^{+}A)w\;\; - \;\; (b_3 - BA^{+}b_1)\|_2 \end{eqnarray*} Therefore, $$ w = (B-BA^{+}A)^{+}(b_3 - BA^{+}b_1) $$ and $$ x = A^{+}b_1+(I-A^{+}A)(B-BA^{+}A)^{+}(b_3 - BA^{+}b_1) $$