How to interpret inverse of a matrix?

Question

I am new to linear algebra, and I am trying to interpret the inverse of a matrix.

If I have the equation $Ax = y$ and I solve for $x$ as $x = A^{-1}b$, plugging $x$ back in as $Ax$ results in either $Ax = y$ or $Ax \neq y$. How should I interpret this? It seems like the inverse gives the correct answer sometimes and sometimes doesn't without any warning.

In regular algebra if I am able to solve $ax = y$ for $x$, $ax = y$ is always true. Is there anyway of knowing whether my resulting $x$ will give me the right answer in advance, without explicitly checking the result of $Ax$ against my original $y$? I thought the inverse operation is supposed to give you back the original input, but inverse matrix multiplication seems strangely inconsistent.

Answer 1

Consider an invertible $n \times n$ matrix $A$ with entries in a field $k.$ We claim that the $n \times n$ matrix $B$ such that $AB = I_{n \times n} = BA$ (where $I_{n \times n}$ is the $n \times n$ matrix with $1$s on the diagonal and $0$s elsewhere) is unique.

Proof. We will assume that there exists another $n \times n$ matrix $C$ such that $AC = I_{n \times n} = CA.$ Using the associativity of matrix multiplication, we have that $B = BI_{n \times n} = B(AC) = (BA)C = I_{n \times n}C = C.$ QED.

Consequently, we can designate the matrix inverse $B = A^{-1}$ of $A.$ Unfortunately, if you are dealing with an $m \times n$ matrix $A$ with entries in the field $k$ for distinct $m$ and $n,$ then $A$ might have a left-inverse $L$ such that $LA = I_{n \times n}$ or a right-inverse $R$ such that $AR = I_{m \times m}$ or neither but not both.

Consider the $3 \times 2$ matrix in your example $$A = \begin{pmatrix} 1 & 0 \\ 0 & 1 \\ 1 & 0 \end{pmatrix}.$$ Observe that this matrix cannot (by the linked post) have a right-inverse; however, it does have a left-inverse $$L = \begin{pmatrix} \frac 1 2 & 0 & \frac 1 2 \\ 0 & 1 & 0 \end{pmatrix}.$$ Ultimately, if we wish to solve $A \mathbf x = \mathbf y,$ we may apply $L$ on the left to obtain $\mathbf x = I_{2 \times 2} \mathbf x = (LA) \mathbf x = L(A \mathbf x) = L \mathbf y;$ however, we cannot conclude from this that $\mathbf x = L \mathbf y$ yields $A \mathbf x = \mathbf y$ because $L$ cannot be both a left- and right-inverse of $A.$

One more comment: the range of $A$ is $\operatorname{span}_k \{\langle 1, 0, 1 \rangle, \langle 0, 1, 0 \rangle \}$ since we have that $$A \mathbf x = \begin{pmatrix} 1 & 0 \\ 0 & 1 \\ 1 & 0 \end{pmatrix} \begin{pmatrix} a \\ b \end{pmatrix} = \langle a, b, a \rangle = \langle a, 0, a \rangle + \langle 0, b, 0 \rangle = a \langle 1, 0, 1 \rangle + b \langle 0, 1, 0 \rangle.$$ Consequently, the vector $\mathbf y = \langle 1, 1, 0 \rangle$ is not in the range of $A,$ i.e., the equation $A \mathbf x = \mathbf y$ has no solution.

Answer 2

Took me some time to figure this out. Basically, you can think as matrix as a linear operation. It is a function that maps numbers to numbers. You can think of the inverse as the 'undo' operation of this matrix. This also means you can't lose information (when operating in the linear space (no non-linear operations))

Lets say your original number is 3, and matrix is 5. $$3 * 5 = 15$$ Inverse of 5 is $1/5$.

$$15 * 1/5 = 3$$ You just undo what you did. In other words, given the results (15) and the inverse ($1/5$), you can re-construct the original number of 3.

However, a non-invertible matrix is 0.

$$3 * 0 = 0$$ (you have lost information) There is no inverse for 0, 1/0 is impossible. Hence, given the results (0) and no inverse, it is impossible to get back to the original number of 3.