From searching on the internet, I learned only a subset of linear transformations preserve angles between vectors. But -
Learning about the SVD - we can geometrically understand as breaking down some matrix A into a three matrices. These matrices can be understood geometrically as a rotation step, then a scaling step, and then another rotation.
Since any matrix can be broken down into these three steps (Since SVD applies to all matrices A?) doesn't that mean that all transformations are simply a rotation, a scaling, and then a rotation, which means the angles are preserved?
Why is this not true? And when do linear transformations preserve angles, and when do they not?
Thanks, A
Let $x$ and $y$ be any two vectors. Then $y - x$ is also a vector and the three vectors $x$, $y,$ and $y - x$ correspond to three sides of a triangle.
Now apply the linear transformation $A$, so $x$, $y,$ and $y - x$ are mapped to $Ax$, $Ay,$ and $A(y - x) = Ay - Ax$, again corresponding to three sides of a triangle.
If the transformation $A$ preserves angles between vectors, it must preserve the angles between each pair of the three vectors $x$, $y,$ and $y - x$, which implies that the triangle corresponding to $Ax$, $Ay,$ and $Ay - Ax$ is similar to the triangle corresponding to $x$, $y,$ and $y - x$, which implies that the three sides of the triangle all grow or shrink in the same proportion; they all scale by the same magnitude. (Note that for similar triangles in geometry the scaling factor can be taken to be positive, even if the triangle is reflected during the scaling.)
Now consider any orthonormal basis of the space and let $x$ and $y$ be an arbitrary pair of basis vectors. If $A$ preserves angles, it must scale $x$ and $y$ by the same magnitude (that is, their scaling factors must have the same absolute value) in addition to any possible rotation. Continue in this way using different basis vectors in place of $y$ until you have shown that all basis vectors are scaled by scaling factors with the same absolute value.
This applies to an inner product space with any finite number of dimensions.
So if we want to preserve angles, the linear transformation must scale all basis vectors by the same absolute value. On the other hand, if the transformation scales all basis vectors by the same absolute value then it transforms each triangle to a similar triangle, and therefore each pair of vectors to a pair of vectors with the same angle between them. So this requirement is not only necessary but is also sufficient.
Applying this to the SVD, the scaling step must be a uniform dilation in all directions composed with zero or more reflections, each of which reverses the direction of a basis vector. This means that if the upper left entry in the matrix $\Sigma$ is $\sigma_1,$ every other entry on the diagonal is either $\sigma_1$ or $-\sigma_1.$
Ultimately you should be able to show that a linear transformation that preserves angles can be decomposed into a uniform dilation, a rotation, and (optionally) a reflection.