Motivation: I have two different matrices in $\mathbb{R}^{1000 \times 2048}$. $A_1$ is coming from an sparse optimization process whose objective is creating as much as zeros in $A_1$. In this sense, I have $A_1$ that has almost 50% zeros in it, while the other, i.e., the original one $A_2$, is not sparse. I plotted the singular value spectrum of them and I got this picture.
Since the spectrum is almost the same for both of them, I am wondering if this means any special thing. Precisely, if we let $A_1=U_1\Sigma V^{\top}_1$ and $A_2=U_2\Sigma V^{\top}_2$ can we say anything about similarities between $A_1$ or $A_2$.
Note: If we can take any conclusion please provide the proof otherwise please mention an example that shows they can be very different even though they have the same spectrum.
So, if they are just close but not exactly the same, I am not so sure there is much that we can say. However, for the purpose of exploration suppose that the singular values of both matrices are the same and we have,
$$A=U_1\Sigma V_1^*$$ $$B = U_2 \Sigma V_2^*$$
One way this could happen is if the two matrices were orthogonally equivalent. We can see this because if $B=QAQ^*$ we have,
$$B=(QU_1)\Sigma(V_1^*Q^*)$$
And since the product of unitary matrices is unitary we have that both $A$ and $B$ share the same singular values. This would also imply that these two matrices are similar since $U^* = U^{-1}$ for unitary matrices. Of course, not all similar matrices share the same singular values.
Furthermore, both directions are not true. That is, while unitary equivalence implies the same singular values, it is not true that sharing singular values implies unitary equivalence.
For a counterexample, suppose $B$ is not square and has singular values $\Sigma$. Let us construct $A=U\Sigma V$ for some unitary $U$ and $V$ so that $A$ is square. But, because $B$ is not square, $A$ and $B$ cannot be unitarily equivalent.
That example of course means that we may not be able to say much at all about matrices with the same singular values.