Suppose there exists some $\vec{b}$ such that $A\vec{x} = \vec{b}$ is unsolvable. Does there exist a $\vec{z}$ such that $A\vec{x} = \vec{z}$?

Question

Problem. Suppose $A$ is a non-zero, square $n\times n$ matrix, such that there exists some column vector $\vec{b}$ such that $A\vec{x} = \vec{b}$ has no solutions. Does there exist some $\vec{z} \neq \vec{0}$ such that $A\vec{x} = \vec{z}$ has a solution? If so, how many solutions exist? Are the columns of $A$ linearly in-dependent or dependent?

While it is clear to me that $span(A) \neq \mathbb{R}^n$ (and therefore the columns of $A$ are linearly depdendent), it isn't clear to me whether there exists some nonzero $\vec{z}$ such that $A\vec{x} = \vec{z}$ is solvable.

I know that $A\vec{x} = \vec{0}$ must have more than the trivial solution. I also know that $\text{RREF}(A)$ must have fewer than $n$ pivots (i.e. at least one free variable). In general, however, is it not the case that such a system may have either no or infinitely many solutions for an arbitrary $\vec{z}$? How do we know that a $\vec{z}$ with a solution (which must, therefore, have infinitely many solutions) exists?

Answer 1

Since $A\ne0$, there is some $i\in\{1,2,\ldots,n\}$ such that the $i$^th column is not null. Let $\vec z$ be the vector whose coordinates are the entries of that column. Then $A\cdot(0,0,\ldots,0,1,0,\ldots,0)$, with the $1$ at the $i$^th position, is equal to $\vec z$. Of course, if $\lambda$ is a scalar, then,$$A\cdot(0,0,\ldots,0,\lambda,0,\ldots,0)=\lambda\vec z,$$and therefore there are infinitely many solutions.

Answer 2

Given $A$ and $\vec{z}$, then there exists a $\vec{x}$ such that

$$A\vec{x} = \vec{z},$$

if $\vec{z} \in \text{Im}(A)$,

where $\text{Im}(A)$ is the subspace spanned by the columns of $A$.

In your example, $\vec{b} \not\in \text{Im}(A)$. As a consequence, as you correctly stated, since there is a vector $\vec{b}$ which is not a member of $\text{Im}(A)$, then for sure $\text{Im}(A)$ is not equal to $\mathbb{R}^n$, or equivalently, the rank of $A$ is less than $n$, or $\det(A) = 0$, or $A$ has at least one zero eigenvalue.

...Or, the columns of $A$ are not linearly independent.

If $m < n$ is the dimension of $\text{Im}(A)$, i.e. $\text{rank}(A) = m$, then there are $\infty^{n-m}$ solutions for the problem $A\vec{x} = \vec{z}$.

Explanation

If $\vec{z} \in \text{Im}(A)$, then there exists real numbers $x_1$, $x_2$, $\ldots$, not all null, such that:

$$\vec{z} = x_1 \text{col}_1(A) + x_2 \text{col}_2(A) + \ldots.$$

But

$$x_1 \text{col}_1(A) + x_2 \text{col}_2(A) = A \vec{x},$$

given that $\vec{x} = [x_1, x_2, \ldots]^\top$, and hence

$$A \vec{x} = \vec{z}.$$

Answer 3

Yes, for example:

$$A = \begin{bmatrix}1&2 \\ 2 & 4\end{bmatrix} \qquad \vec{b} = \begin{bmatrix} 0 \\ 1 \end{bmatrix} \qquad \vec{z} = \begin{bmatrix} 3 \\ 6 \end{bmatrix}$$

You will be able to answer all your questions if you study a bit more linear algebra. In short, you will learn how to visualize the columns of a matrix with vectors in $n$ dimensions (where $n$ is the number of rows of the matrix). The most relevant theorem is the Rank-nullity theorem, which states that

$$\text{rank}(A) + \text{nullity}(A) = n$$

Where the $\text{rank}(A)$ of the matrix is the dimensionality of the space spanned by its columns (or rows), and the $\text{nullity}(A)$ is the dimensionality of the space spanned by the possible solutions of $x$ vectors to $Ax = 0$.