What is the geometric interpretation of the rowspace?

Question

I am missing something essential in my understanding of matrices as linear mappings (or matrices in general). To explain, I think I have a decent intuition for columnspace of a matrix. Basically, the columnspace can be thought of as the set of points other than the origin which can be mapped by the matrix of a linear mapping (I like to think of matrices in general as linear mappings).

What I have trouble with is the significance and geometric interpretation of the rowspace of a matrix. By the definition, I can see that the rowspace and columnspace are analagous; the rowspace is merely the columnspace of the transpose of the matrix of the linear mapping. However, I cannot explain this anymore intuitively than above. To quote Albert Einstein, "if you don't understand it simply, you don't understand it well enough". I am looking for a simple explanation (or geometric interpretation that will contribute to my imagination of linear mappings).

How would you explain this concept?

To give you some context, I ran into problems when my course started to discuss the following:

For an $n \times n$ matrix $P$, the following are equivalent:

(1) The columns of $P$ form an orthonormal basis for $\mathbb{R}^n$

(2) $P^T = P^{-1}$

(3) The rows of $P$ form an orthonormal basis for $\mathbb{R}^n$.

I understand the algebra perfectly, but I seek a level of intuition so that I would not need to fiddle around with algebra to discover these facts.

How would you explain the above concepts intuitively or geometrically, without the need for algebra to make a case?

Many thanks in advance.

Answer 1

As Ted Shifrin states in his comment to your question, the rowspace of a real matrix is the orthogonal complement of its nullspace. In fact, we have the following relationships for any (not necessarily square) real matrix $A$: $$\mathscr{C}(A^T)=\mathscr{N}(A)^\perp \\ \mathscr{N}(A^T)=\mathscr{C}(A)^\perp.$$ There is an equivalent pair of statements about a linear map, its adjoint map and annihilators of subspaces.

One consequence of these relationships is that the rowspace of a matrix is a maximal-dimension subspace of the domain for which the linear map defined by the matrix is injective. Another possible way to view the rowspace is as the space of representative elements of the quotient space $\mathbb R^m/\mathscr{N}(A)$, where $m$ is the number of columns of $A$. In light of this, the rowspace of a matrix can be considered the “natural” preimage of its column space.

Answer 2

This is in a sense trivial but also maybe not something you'd immediately think of.

Consider a square matrix $A$ (though it doesn't have to be).

The columns are a basis for the output (column) space. The image of the cartesian grid in the input space is an affine, probably skewed (unless the matrix is orthogonal) grid in the output space.

The grid lines are parallel to these basis vectors (the columns of $A$).

Consider the image of the unit sphere in the input space. This will be some ellipsoid in the output space.

Choose a particular cartesian axis in the output space, say the z (3rd) axis.

What point on the ellipsoid image "sticks out" furthest along the z axis? i.e., what point has the greatest z coordinate?

That point will lie on the span of the vector that is the image under $A$ of the 3rd row of $A$ used as an input to $A$. i.e., the span of $A\vec{v}$ where $\vec{v}^T$ is the 3rd row of $A$.

So if the 3rd row of $A$ is $\left[a\ b\ c\right]$, then the vector in the output space whose tip is at the $\left[a\ b\ c\right]$ point of the (skewed) output space grid will run through that point of the ellipse.

One way to explain this is that the "Jacobian" of $\vec{f}(\vec{x}) = A\vec{x}$, of $\vec{f}$ with respect to $\vec{x}$, is $A$. So the direction in the input space of greatest increase of the mth output dimension (coordinate) is the mth row.

Looking at the image of the unit sphere of the input space corresponds to limiting yourself to a fixed "step size" to take along the gradient.