Let $M_n$ be the space of $n \times n$ real matrices, and consider the following equivalence relation on $M_n$:
$A \sim B$ if there exist $Q \in O(n)$ such that $A=QB$.
Can we characterise nicely the equivalence classes of this relation?
By comparison, the usual "orthogonal equivalence", i.e. $A \sim B \iff A=QBU$ for some $Q,U \in O(n)$ gives rise to the notion of SVD-i.e. each equivalence class corresponds to a specific finite sequence of singular values. Thus, the singular values are the "invariants" which classify the equivalence classes.
Is there any sensible way to associate a list of classifying invariants to the new relation in a similar way?
Of course, we shall need "more invariants" to distinguish between different classes: It will be something like "the same singular values+something else".
In particular, we have $A \sim B \implies \ker A=\ker B$, but for invertible matrices this does not really add any information.
Even finding some non-trivial sufficient conditions for when $A \sim B$ would be interesting: Having the same singular values is certainly necessary, but is far from sufficient, which can be checked directly even at dimension $2$. ($\Sigma$ and $\Sigma Q$ are not equivalent generically).
Two matrices $A,B \in M_n(\mathbb{R})$ are equivalent under your relation if and only if $A^T A = B^T B$. In one direction, if $A = QB$ then
$$ A^T A = (QB)^T (QB) = B^T Q^T Q B = B^T B. $$
In the other direction, if $A^T A = B^T B$ then we can use the polar decomposition to write $A = QP, B = UP$ where $P = \sqrt{A^T A}$ and $Q,U \in O(n)$ are orthogonal and then
$$ A = QP = (QU^{-1}) UP = (QU^{-1})B $$ and since $QU^{-1}$ is orthogonal, we have $A \sim B$. In fact, if you denote by $X \subseteq M_n(\mathbb{R})$ the subspace of positive semidefinite matrices, the map $[A] \mapsto A^TA$ gives you a homeomorphism between the orbit space $M_n(\mathbb{R}) / \sim$ and $X$. This map is well-defined, one-to-one and continuous by the property of the quotient topology with a continuous inverse $P \mapsto [\sqrt{P}]$.