So I initially learned about this topic around a year ago in my second semester of linear algebra, but try as I might, I could never figure out how the heck the formula does what’s it supposed to do. I asked a lot of questions about it and I think I kind of understood it, but not completely. I just started thinking about it again and I was reading the wikipedia page on it: https://en.m.wikipedia.org/wiki/Singular_value_decomposition Looking at that article, the most intuitive explanation to me is that SVD breaks the linear transformation up into 3 simpler transformations — scaling, reflecting, and rotating. It makes sense that there would be a way to do that and it sounds very similar to what polar coordinates do for vectors. The wiki page mentioned that SVD is similar to polar decomposition, which I didn’t know was a thing, though it’s not surprising that it is. I tried to read the page on that, but it just started explaining it in terms of SVD, which is what I was trying to understand in the first place. It also mentioned that it’s a generalized version of eigendecomposition. That also makes sense, since the formulas seem kind of similar, but I don’t understand how exactly it does this. Eigendecomposition makes more sense to me, but something abiut SVD just keeps throwing me off.
It might be help for me to think about it like this: Let’s say I have some arbitrary linear transformation written as a matrix and you want to represent as the product of scaling matrix, a reflection matrix, and a rotation matrix and pretend we don’t know about SVD; it’s not been invented yet. How might we go about figuring out a systematic way of doing this? I imagine we could figure out the geometry of the transformation using matlab or another program and then separately find each matrix to represent each separate piece of the transformation, but that wouldn’t be very efficient and it would rely on software to do the hard part for us.