If an overdetermined system of linear equations has a solution, can it be found by ignoring some of the equations?

Question

For simplicity, take $A$ to be a $3\times 2$ matrix, and $b\in\mathbb{R}^3$. Assume the linear system of 3 equations $Ax=b$ has at least one solution. If this is the case, is it always possible to find this by deleting the last linear equation and solving the remaining 2 variable linear system?

Answer 1

Not always. You need to make sure that the equation you're deleting is actually implied by the other equations.

Consider the system

$$\begin{pmatrix}1&0\\1&0\\0&1\end{pmatrix}x = \begin{pmatrix}0\\0\\17\end{pmatrix}.$$

The only solution is $x = \begin{pmatrix}0\\17\end{pmatrix}$, but if you delete the final row of $A$ and $b$ then every $x = \begin{pmatrix}0\\t\end{pmatrix}$ would be a solution.

Answer 2

Hint: It depends on the rank of the coefficient matrix and on the rank of the total matrix (coeff. + b), and on that of the minors obtained by subtracting one of the equations.

You can find all the explanations in any introductory lesson to Linear Algebra.