I have a matrix $A_{3\times2}$ , $U_{3\times2}$ , $V_{2\times2}$.
I was asked to find the singular values, which I did with by multiplying $U'AV$.
I got the $U'$ by following this rule: Since $U$ is an orthogonal matrix, the inverse of $U$ is equal to its transpose.
But now the question:
If I didn't follow the rule of transposing $U$, then I would have a matrix multiplication of dimension $(3\times2)(3\times2)$ which doesn't work.
Any suggestions, tips on what am I missing here?
$$A=U\Sigma V'\implies AV=U\Sigma\implies U'AV=\Sigma$$.
Sizes - $A_{m\times n}, U_{m\times m},\Sigma_{m\times n},V_{n\times n}$.
All multiplications are valid, and you got the singular values on the diagonal of $\Sigma$
When you get a non square matrix $U$, you need to "complete" it to be a square matrix. How? Just add random (almost) vectors to it. Just be sure they keep your matrix columns orthonormal (Gram Shcmidt to the rescue). Now we need to fix $\Sigma$. Just add rows of zeros to it (yes, it is that simple) and you are done. As the comment suggests, this is just for mathmatical equality, and you dont have to do this in order to complete the SVD decomposition.