The Singular Value Decomposition of a matrix A satisfies $\mathbf A = \mathbf U \mathbf \Sigma \mathbf V^\top$.
I wanted to clarify how this diagram works. I understand this diagram if the "inputs" into $A$ are two orthogonal basis vectors, but how come the outputs are also orthogonal? How would you know this would be true for any rectangular matrices?
Or is it more so saying "If we know the image of two vectors we can define the linear map by matrix $A$, and we choose two special vectors such that their images will be $\sigma_1 u_1$ and $ \sigma_2 u_2 $? But then how would you know that these two special vectors will be orthogonal? (So that ( V^T ) makes sense?)
Thanks,