Understanding Linear Algebra Geometrically - Reference Request

Question

I know geometry and I know linear algebra but when I understand a linear algebraic concept geometrically, my head just explodes and things just become so much clearer and easier to understand...not to mention easier to remember or figure out its properties and explain them to others.

Here are a few examples.

1. Orthogonal matrices - If you think of an orthogonal matrix as a rotation then some of its properties are obvious. Orthogonal matrices are always invertible because rotations can simply be reversed. They always preserve the Euclidean norm because rotating a vector doesn't change its length. Orthogonal matrices forming a group is also easy to see because it is easy to see them satisfying the group axioms.

2. Determinant - The determinant of a linear transformation can be understood as follows. Start with the (chosen) basis of your domain. It forms a parallelepiped. Call it $P$. It has a certain volume $V(P)$. Now apply your linear transformation $T$ to the chosen basis. A new parallelepiped $T(P)$ is formed and its volume in the range space (embedded in the codomain) is now $V(T(P))$. The determinant (in absolute value) is the ratio of the new volume to the old one. This intuitively explains, for example, why the determinant is zero for non-invertible transformations. The dimension of such a transformation will always be strictly less than the dimension of the domain/codomain so the volume of the transformed parallelepiped will always be zero. I always imagine a parallelepiped in $\mathbb{R}^3$ collapsing onto a plane. This also explains why the determinant of an orthogonal matrix is always $\pm1$ because rotating a parallelepiped won't change its volume. In addition, it kind of helps with the Jacobian determinant and why is the Jacobian "necessary" when transforming variables.

3. Singular value decomposition - Every matrix having an SVD says the fantastical fact that any linear transformation can be considered a rotation, then a dilation (different directions by different factors), and then a rotation again.

4. Projection matrices - Imagine an arbitrary vector's shadow onto a line or a plane. I imagine a vector collapsing onto its shadow and properties like $P^2=P$ are immediate for any projector operator $P$. Take this and run with it.

My question is, can anyone point to some good reading material where a geometric interpretation of various linear algebra concepts is offered?

This could be anyone's class/teaching notes, published papers, something from recreational mathematics, or just a good book.

Answer 1

The vector $y-x$ connects the end of $x$ to the end of $y$.

Answer 2

Here is something I use when teaching an introductory linear algebra course.

If $V$ is a vector space over a field $\mathbb{F}$, then a line in $V$ can be considered as any set of the form $$ \mathcal{L} = \{ x+\alpha y \mid \alpha \in \mathbb{F}\} \subseteq V, $$ in which $x$ and $y$ are elements of $V$ (note that this includes the degenerate case of a single point if $y=0$).

If $T:V \longrightarrow W$ is a linear transformation, then $$ T(\mathcal{L}) = \{ T(x)+\alpha T(y) \mid \alpha \in \mathbb{F}\}. $$ Thus, linear transformations map lines to lines. (This is the explanation I give as to why the word 'linear' is used in 'linear transformation'.)

It is worth noting that this condition is not sufficient for linearity: affine transformations of the form $x \longmapsto Ax + b$ will also map lines to lines.

Answer 3

An intuitive view point of the Primary Decomposition Theorem for Eigenspaces is to consider an orthonormal basis. For example, for the vector space $\mathbb{R}^3$ spanned by $\left\{\mathbf{\hat{i}}, \mathbf{\hat{j}}, \mathbf{\hat{k}}\right\}$, the eigenspaces are $\mathcal{E}_1 = \mathcal{L}\left(\left\{\mathbf{\hat{i}}\right\}\right)$, $\mathcal{E}_2 = \mathcal{L}\left(\left\{\mathbf{\hat{j}}\right\}\right)$, and $\mathcal{E}_3 = \mathcal{L}\left(\left\{\mathbf{\hat{k}}\right\}\right)$.

By the Primary Decomposition Theorem for Eigenspaces, $$\mathbb{R}^3 = \mathcal{E}_1 \oplus \mathcal{E}_2 \oplus \mathcal{E}_3$$

And thus any vector must be uniquely expressed by the eigenvectors of the space, in this case $\mathbb{R}^3$. For an orthonormal basis, it is easy to see why the direct sum should span the entire space, since the eigenspaces correspond to the linear span of each basis element.

It should then be easier to have an intuition for what is happening for any eigenbasis of $V$ that is not necessarily orthonormal.

Answer 4

It might be slightly off-topic since you are searching for reading material, but the 3blue1brown YouTube channel is dedicated to this sort of geometric intuitions. More specifically, it presents abstract concept with geometric animations, spanning from neural networks to Fourier transform.

Although it is not restricted to such matter, it has an excellent playlist called Essence of linear algebra which adresses what you seek.

Answer 5

I'm the same way, you pretty much described my experience with Linear Algebra suddenly 'clicking' when it's explained geometrically. I found a really good book that treats alot of different LA concepts in this way, it's Numerical Linear Algebra by Trefethen.

Check out chapter 4 for a geometric breakdown of Singular Value Decomposition!

Answer 6

Although old and quite hard to obtain, Fekete's Real Linear Algebra is probably the perfect book for this. With a hit in generality with it's focus on $\mathbb{R}^n$, it delivers quite a geometric take on the whole subject you can hardly find elsewhere, with even an Applications (to geometry) section in every chapter.