Solution to linear system of equations

231 Views Asked by At

Notation. Let $y$, $a$, and $b$ be $n\times 1$, $p\times 1$, and $q\times1$ real vectors. Let also $X$ and $Z$ be $n\times p$ and $n \times q$ real matrices.

Suppose that there is no solution, $a$, to $y = X a$.

Question. What are the conditions on $Z$ such that $y = Xa + Zb$ has no solution for each choice of $b$?

Context. I came across this problem in the context of linear regression. The fact that $y=Xa$ has no solution can be interpreted as "no hyperplane can perfectly fit the data". I am analysing an extension of this problem which has lead me to the need of finding something similar for "$y = Xa + Zb$ has no solution".

1

There are 1 best solutions below

1
On BEST ANSWER

You are essentially asking, when does $y = \left[X\ Z\right]b$ have no solution $b$? Or in other words, when is $y$ not in the direct sum of the column spaces of $X$ and $Z$?

An obvious necessary, but not sufficient, condition on $Z$ is that $y = Zu$ has no solution $u$. I don't think there's a nice complete set of conditions on $Z$; in practice you can row-reduce $[X\ Z]$ and see if $y = [X\ Z]b$ is a consistent linear system, or equivalently, solve the least-squares problem $$\left[\begin{array}{cc} X^TX & X^TZ\\ Z^TX & Z^TZ\end{array}\right]b = \left[\begin{array}{c}X^T\\Z^T\end{array}\right]y$$ for $b$ and see if the residual is zero.