What is the intuition behind why a rank deficient matrix does not have an inverse?

Question

Suppose that we have a $p$ dimensional square matrix $A$ whose rank is less than $p$. We know that such a matrix cannot have an inverse and there are several different ways to prove that the $A$ does not have an inverse.

However, I am struggling to obtain an intuition behind why the inverse does not exist. I considered the following ideas to generate an intuition but failed to do so.

1. The matrix $A$ can be viewed as a set of transformations such as scaling, translation, rotation etc. Thus, when we apply $A$ to a vector in $p$ dimensions it always maps the $p$ dimensional vector to a vector in a subspace spanned by $A$ if it is less than full rank. Lack of an inverse implies that we cannot reverse the transformations. Why not?

2. The columns of $A$ span only a subspace of $R^p$ if it is less than full rank. Thus, a transformation such as $A y$ takes a vector from $R^p$ to a vector that always belongs to that subspace. The above viewpoint did not help in obtaining an intuition either.

Is there a way to obtain an intuition as to why a rank deficient matrix does not have an inverse?

Answer 1

The row reduced echelon form of your matrix will have some rows of zero at the bottom , which is not invertible.

An invertible matrix when reduced to its row reduced echelon form becomes the identity matrix.

Answer 2

"Invertible" when talking about linear transformations means "reversible". In other words, a linear transformation (and the corresponding matrix in a given basis) is invertible iff it is possible, given an output, to figure out exactly what the input was.

A rank deficient linear transformation will collapse at least one dimension, meaning each output could be the result of any of a number of different inputs. Specifically, it will have a non-trivial kernel, so multiple different inputs result in the output $\vec 0$.

Answer 3

The way i learned is was like this: You should see the Det function as sending a matrix (of dim=n),to the oriented n-dim volume it's column vector's span. if this is zero, this will be a n-1 dim hyper-surface. Thus losing surjectivity into its image, we know injectivity and surjectivity are equivalent for finite dim square matrices. So it can't be invertible.

Answer 4

This stems from the fact that a mapping $f:V\to V$ cannot possess any left inverse if it is not injective and it cannot possess any right inverse if it is not surjective:

• if $f$ is not injective, i.e. if $f(u)=f(v)$ for some $u\ne v$, then $f$ cannot have any left inverse, otherwise we would have $u=(f^{-1}\circ f)(u)=f^{-1}(f(u))=f^{-1}(f(v))=(f^{-1}\circ f)(v)=v$, which is a contradiction;
• if $f$ is not surjective, i.e. if there is some member $w$ of $V$ that lies outside $f(V)$, then $f$ cannot possess any right inverse, otherwise we would have $w=f(f^{-1}(w))\in f(V)$, which is a contradiction.

Now, if a square matrix $A$ is rank deficient, its columns are linearly dependent. Therefore $Au=0$ for some nonzero vector $u$. In other words, $A$ maps both $u$ and $0$ to $0$. Hence $A$ has not any left inverse, because the mapping $x\mapsto Ax$ is not injective. (If you invert back, what should $A^{-1}0$ be? $u$ or $0$?)

Also, as $A$ is rank deficient, its column space $A$ is a proper subspace of the ambient space. Hence $A$ has not any right inverse, because the mapping $x\mapsto Ax$ is not surjective. (If $w$ lies outside the column space of $A$ and it has an inverse image, then $w$ itself is the image of its own inverse image, hence $w$ also lies inside the column space of $A$. How paradoxical!)