Geometrically explained, why do Linear Transformations Take a circle to an ellipse

Question

Short Version: How can it be geometrically shown that non-singular 2D linear transformations take circles to ellipses?

(Also, its probably important to state I'd prefer an explanation that doesn't use SVD, as I don't really understand it yet...although I see it everywhere)

Long Version: Let's use the definition of an ellipse as being a circle stretched in two perpendicular directions. The two directions its stretched in will correspond to the two axes of the ellipse.

We begin by defining the circle as the endpoints of all possible 2D unit vectors. The ellipse (or at least I"m TOLD it's an ellipse) is the shape drawn by the endpoints of all these vectors after they have all been transformed in the same way (aka multiplied by the same nonsingular matrix $A$).

1. For linear transformations represented by diagonal matrices, it's easy to see. We're just stretching the circle in the X and Y directions.

2. For linear transformations represented by symmetric matrices...its a little harder, but I can see the transformation because the eigenvectors of the symmetric matrix are perpendicular, and if we change to a basis where those eigenvectors are the basis vectors, the transformation can be represented by a diagonal matrix (as for WHY symmetric matrices can be decomposed this way I don't yet really understand - but for the purpose of this question I'm just accepting that they can; I'm accepting that the eigenvectors of symmetric matrices are perpendicular to one another).

   So, just like diagonal matrices, symmetric matrices also correspond to stretching a unit circle in perpendicular directions - but unless the symmetric matrix is diagonal, these are perpendicular directions different from the X and Y directions.

3. Buuut...what about for nonsymmetric matrices?

Thanks!

---

EDIT:

I've now learned of the polar decomposition of any real matrix, and that provides a beautiful explanation for why any real matrix takes a circle to an ellipse!

$A=QS$, where $Q$ is an orthogonal matrix (rotation) and $S$ is a symmetric matrix (stretching in the direction of the eigenvectors).

The symmetric matrix will definitely correspond to making an ellipse (since it scales in orthogonal directions, although perhaps not our regular $x$ and $y$ directions) and all the orthonormal matrix will do is rotate this ellipse.

However, all the explanations I've seen so far that PROVE that polar decompositions of real matrices are always possible use an algebraic explanation instead of a geometric one...so they aren't really what I'm looking for.

Thanks again!

Answer 1

Every real square matrix has a polar decomposition into the product of an orthogonal matrix $U$ and a positive-semidefinite (symmetric) matrix $P$. If the original matrix is nonsingular, then $P$ is positive-definite. In 2-D, orthogonal matrices represent either rotations or reflections, which are both isometries, so they don’t affect the shape of the transformed circle. As you’ve mentioned, $P$ can be orthogonally diagonalized, so it represents a stretch in some set of perpendicular directions.

The existence of this decomposition is equivalent to the existence of the SVD, but can be shown without relying on the latter. In a similar vein, the SVD decomposes the matrix into the product of a rotation or reflection, a scaling, and another rotation or reflection.

You might also have a look at the Steiner generation of an ellipse. This uses intersecting line segments drawn between points on the sides of a parallelogram to generate ellipses, including circles. Affine transformations preserve incidence relationships (the image of the intersection of a pair of lines is the intersection of the lines’ images) and maps paralellograms to parallelograms, so the image of an ellipse under an affine transformation is another ellipse.

Answer 2

The equation of a circle is $x^2 + y^2 = r^2$, or in terms of vectors $(x,y) \pmatrix{x\cr y} = r^2$ An invertible linear transformation $T$ takes $\pmatrix{x\cr y}$ to $\pmatrix{X\cr Y} = T\pmatrix{x\cr y}$. Thus $\pmatrix{x\cr y\cr} = T^{-1} \pmatrix{X\cr Y}$, and $(x,y) = (X, Y) (T^{-1})^\top$. The equation becomes $$(X, Y) (T^{-1})^\top T^{-1} \pmatrix{X\cr Y} = r^2 $$ Note that $(T^{-1})^\top T^{-1}$ is a real symmetric matrix, so it can be diagonalized, and its eigenvalues are positive.

Answer 3

This answer really demands illustrations, but I'm too lazy to draw them. I encourage you to draw your own illustrations as you read. If you like them enough to share, leave a comment—I'll make the answer community wiki so you can add them!

The outermost points

Take an invertible linear map $A$ from one two-dimensional space to another. This map turns the unit circle $\mathcal{S}$ into some mystery shape $A\mathcal{S}$, which we want to understand. If we take a unit vector $\mathbf{u}$ and spin it all the way around the unit circle, $\|A\mathbf{u}\|^2$ can't increase forever, because $\mathbf{u}$ will eventually come back to where it started. That means there must be places where $A\mathcal{S}$ reaches a maximum distance from the origin. Let's focus on those outermost points.

Suppose $A\mathbf{u}_0$ is one of the outermost points of $A\mathcal{S}$. Choose a moving unit vector $\mathbf{u}_t$ that travels at unit speed along the unit circle, passing through $\mathbf{u}_0$ at time $t = 0$. If we graph $\|A\mathbf{u}_t\|^2$ with respect to $t$, we get a smooth curve. At $t = 0$, the graph has a maximum, so its slope must be zero. That's a clue about what $A\mathcal{S}$ looks like near $A\mathbf{u}_0$. Let's study it more carefully.

Over short distances, moving at unit speed along a circle is very close to moving at unit speed along a straight line [1]. In symbols, $$\mathbf{u}_t \approx \mathbf{u}_0 + t\mathbf{w},$$ where $\mathbf{w}$ is a unit vector perpendicular to $\mathbf{u}_0$. This means that when $t$ is near zero, the graph of $\|A\mathbf{u}_t\|^2$ is very close to the graph of $\|A(\mathbf{u}_0 + t\mathbf{w})\|^2$. In particular, at $t = 0$, the slope of the graph of $\|A(\mathbf{u}_0 + t\mathbf{w})\|^2$ matches the slope of the graph of $\|A\mathbf{u}_t\|$—which is zero.

The function $\|A(\mathbf{u}_0 + t\mathbf{w})\|^2$ is convenient, because we can break it down using dot product algebra: $$\begin{align*} \|A(\mathbf{u}_0 + t\mathbf{w})\|^2 & = \|A\mathbf{u}_0 + tA\mathbf{w}\|^2 \\ & = (A\mathbf{u}_0 + tA\mathbf{w}) \cdot (A\mathbf{u}_0 + tA\mathbf{w}) \\ & = (A\mathbf{u}_0 \cdot A\mathbf{u}_0) + 2(A\mathbf{u}_0 \cdot A\mathbf{w})\,t + (A\mathbf{w} \cdot A\mathbf{w})\,t^2. \end{align*}$$ Now we see that $\|A(\mathbf{u} + t\mathbf{w})\|^2$ is a quadratic function. Its graph is a parabola. Since know the graph has slope zero at $t = 0$, the coefficient of $t$ must be zero. In other words, $A\mathbf{u}_0 \cdot A\mathbf{w} = 0$, which means $A\mathbf{u}_0$ and $A\mathbf{w}$ are perpendicular.

Let's review. We started by looking at $A\mathbf{u}_0$, one of the outermost points of $A\mathcal{S}$. Then we chose [2] a unit vector $\mathbf{w}$ perpendicular to $\mathbf{u}_0$. By thinking about what a maximum looks like, we concluded that $A\mathbf{w}$ was perpendicular to $A\mathbf{u}_0$. In summary:

If $A\mathbf{u}_0$ is one of the outermost points of $A\mathcal{S}$, then $A$ sends every unit vector perpendicular to $\mathbf{u}_0$ to a unit vector perpendicular to $A\mathbf{u}_0$.

The longest axes

Each unit vector $\mathbf{u}$ has a counterpart $-\mathbf{u}$ on the opposite side of the unit circle. Since $A(-\mathbf{u}) = -A\mathbf{u}$, it follows that every point $A\mathbf{u}$ on $A\mathcal{S}$ has a counterpart $-A\mathbf{u}$ on the opposite side of $A\mathcal{S}$. If $A\mathbf{u}$ is an outermost point, then $-A\mathbf{u}$ is too, because a vector and its opposite always have the same length. We've discovered that the outermost points of $A\mathcal{S}$ come in opposing pairs. Each pair sits on a line through the origin: a longest axis of $A\mathcal{S}$.

This point of view is convenient, because linear maps send lines to lines. If $\mathcal{L}$ is the line along a vector $\mathbf{v}$, then $A\mathcal{L}$ is the line along the vector $A\mathbf{v}$. We can use that idea to make our summary statement even more geometric. For brevity, I'll write $\text{line}^0$ to mean a line through the origin, and I'll write $\mathcal{L}^\perp$ to mean the $\text{line}^0$ perpendicular to $\mathcal{L}$.

If $A\mathcal{L}$ is one of the longest axes of $A\mathcal{S}$, then $\mathcal{L}$ and $\mathcal{L}^\perp$ stay perpendicular when you apply $A$. In other words, $A\mathcal{L}$ and $A\mathcal{L}^\perp$ are perpendicular.

We can use this idea to draw a picture of $A\mathcal{S}$. First, pick a $\text{line}^0$ $A\mathcal{L}$ which is one of the longest axes of $A\mathcal{S}$. Since we're working in two dimensions, we can use $\mathcal{L}$ and $\mathcal{L}^\perp$ as the axes of a coordinate grid. When we apply $A$ to this grid, the axes stay perpendicular; the only kind of distortion we can get is stretching along the axes. If you draw a circle on a coordinate grid and then distort it by stretching along the axes, you get an ellipse! That means $A\mathcal{S}$ is an ellipse.

We now see that in most cases, $A\mathcal{S}$ only has one longest axis: the major axis of the ellipse. The exception is when the stretch factors along both axes happen to be the same, making $A\mathcal{S}$ a circle.

[1] If you've ridden a bike or driven a car, imagine locking the handlebars or steering wheel just a tiny bit off center. If you travel a long distance this way, you'll go in a huge circle, but over short distances it will be hard to tell you're not going straight.

[2] In our original reasoning, $\mathbf{w}$ was determined by the direction $\mathbf{u}_t$ was moving at $t = 0$. However, we're the ones who chose $\mathbf{u}_t$, and we could make it move in any direction.

The next dimension

The same reasoning works in three dimensions, where it shows that applying an invertible linear map $A$ to the unit sphere $\mathcal{S}$ turns it into an ellipsoid. Like before, we see that if $A\mathcal{L}_1$ is one of the longest axes of $A\mathcal{S}$, then $\mathcal{L}_1$ and $\mathcal{L}_1^\perp$ stay perpendicular when you apply $A$. The only complication is that $\mathcal{L}_1^\perp$ is a $\text{plane}^0$ instead of a $\text{line}^0$. The trick is to realize that $A$ is a linear map from $\mathcal{L}_1^\perp$ to $A\mathcal{L}_1^\perp$. In other words, it's a map from one two-dimensional space to another—so our two-dimensional reasoning applies. If $A\mathcal{L}_2$ is one of the longest axes of $A\mathcal{S}$ within $A\mathcal{L}_1^\perp$, and $\mathcal{L}_3$ is the $\text{line}^0$ perpendicular to $\mathcal{L}_2$ within $\mathcal{L}_1^\perp$, then $A$ keeps $\mathcal{L}_2$ and $\mathcal{L}_3$ perpendicular. We end up with three perpendicular $\text{lines}^0$: the longest axis $\mathcal{L}_1$, the second-longest axis $\mathcal{L}_2$, and the third-longest axis $\mathcal{L}_3$. These lines stay perpendicular when we apply $A$.

We can extend this reasoning to any finite number of dimensions. Given an invertible linear map $A$ from one $n$-dimensional space to another, we pick one of the $\text{lines}^0$ that $A$ stretches the most, focus on the $(n-1)\text{-plane}^0$ perpendicular to it, and repeat. We end up with perpendicular $\text{lines}^0$ $\mathcal{L}_1, \mathcal{L}_2, \ldots, \mathcal{L}_n$ which stay perpendicular when we apply $A$, and are ordered from most to least stretched. You asked for an answer that doesn't use singular value decomposition, but the joke's on you: if you try to understand why applying a linear map to a sphere turns it into an ellipsoid, you'll inevitably end up understanding singular value decomposition.

Answer 4

The answers on this thread are quite insightful but I am attempting here a rather very geometric answer, as the OP demanded so. For this I am going to use another interesting geometric interpretation of linear transformation (which is easier to imagine)

An alternate geometric interpretation of (dimension-preserving) linear transformation is that it's a transformation such that any line in original space is always transformed into a line and origin is not shifted.

Now imagine a freshly made pizza base/crust. Even more, let's make grids on it.

Our job is now to transform it's shape such that all the grid lines (or any possible line) remain lines. So you see at best what we can do is *stretch it with equal forces on opposite sides. You are free to choose where to stretch and Of course we can simply rotate it also after or before all the stretching.

It's not difficult to imagine that we can only get an ellipse (or bigger circle if we stretched from all sides with equal force).

Other interesting points: (1): The direction of stretch is eigen vector and extent to which it is stretched is eigen value; (2): Each rectangle from the gridlines is now parallelogram.