Can the equation $Ax$, for a matrix $A$ and a vector $x$, ever return a vector that is not in the column space of $A$?

Question

If $Ax = b$, where $A$ is a matrix and $x$ and $b$ are vectors, then can $b$ ever be a vector that isn't in the column space of $A$ (i.e. a vector that is not a linear combination of the columns of $A$)?

I feel that the answer is "no, it can't", and I also feel that the fact that the answer is no is exactly why the idea of column space is so powerful. But I haven't been able to prove it to myself and so I'm wanting to be sure. Any help is appreciated!

Apologies, I accidentally used two variables $y$ and $b$ when I meant to only use one. They are the same thing.

Answer 1

Your answer No is correct. Indeed, the vector $b$ is always a linear combination of the columns of $A$. In particular writing the $i$-th coordinate of $x$ as $x_i$ (which is a scalar) and the $i$-th row of $A$ as $A_i$, then $b = \sum_i x_i A_i$. You can check this for yourself: For each $\ell$, the $\ell$-th coordinate of $b$ is $\sum_i x_i A_i$ (the $x_i$s are the same for all such $\ell$)

Answer 2

I take it that y and b are the same thing here. To answer your question, Ax=b only if b is in the column space of A.

If b is not in the column space of A, the best you can do is find an x, which gets closest to "being a solution".

It turns out that such an x satisfies $x  =  (A^{T}A) ^{-1}A^{T}b$ (this is called least squares).