I'm still having trouble understanding the separation between diagonalization and singular value decomposition. I am working with complex matrices so for an arbitrary matrix, I am able to write it as $A = U\Sigma V^\dagger$
My questions are
1) Under what conditions do we have $U = V$? I believe this corresponds to diagonalization so it must be that $A$ is square and $A$ is normal i.e $A^\dagger A = AA^\dagger$. Is this correct or is there a weaker condition?
2) Under what conditions are $dim(U) = dim(V)$? Is it true if $A$ is square that we can achieve this with $dim(U) = dim(V) = dim(A)$?
If $A$ is an $m\times n$ matrix, then also $\Sigma$ is $m\times n$, so $U$ must be $m\times m$ and $V$ is $n\times n$.
So if you want that $U$ and $V$ have the same shape, then $A$ must be square.
You're right in thinking that having $U=V$ implies the (square) matrix $A$ is normal: actually it must be Hermitian. If the SVD is $A=U\Sigma U^\dagger$, then $A^\dagger=U\Sigma^\dagger U^\dagger$; since by definition $\Sigma$ is real, we have $\Sigma=\Sigma^\dagger$, so $A^\dagger=A$.
However, here we see a necessary condition so that a unitary diagonalization of $A$ can be a SVD: the eigenvalues of $A$ must be real and nonnegative, that is, $A$ is positive semidefinite.