Solving a System of Equation which wont give perfect answers

Question

I have a system of equations that is used to predict a response. I know the answers and I am solving for the X and Ys to solve the equations. The problem is that I know that there will never be a perfect solution to the equations as they counteract each other, though I want the answers to give a value somewhat close to the actual answers. Any suggestions on ways to do that besides numerically just punching in numbers and seeing if they are close? Thanks!

Answer 1

instead of solving $A\vec{x} = \vec{b}$, you can solve the linear least squares formulation of the problem, $$ A^T A \vec{x} = A^t \vec{b}, $$ which is guaranteed to have the solution. In the geometric projection sense, that will be the closest possible answer to the true solution of the system.

Meaning to say, geometrically, the solutions of $A \vec{x} = \vec{b}$ are representations of $\vec{b}$ in the column space of $A$, which is not always possible. However, you can always orthogonally project $\vec{b}$ onto the column space of $A$ and the linear least squares problem will get you the corresponding solution.

A typical technique for doing this numerically is the singular-value decomposition.