Let $M\in \operatorname{Mat}_{d\times d}(\Bbb{C})$ and define $\|M\|:=\sqrt{\sum_{i,j} |M_{i,j}|^2}$. Let $X,Y\in \operatorname{GL_{d\times d}}(\Bbb{C})$ s.t. $X^HX=I, Y^HY=I$. I want to show that $\|XMY\|=\|M\|$
My Idea was the following. First we know that $\langle AX,Y\rangle=\langle X, A^HY\rangle$ and $I^H=I$. Therefore $$\begin{align}\|XMY\|^2&=\langle XMY, XMY\rangle \\&=\langle X^HXM,MYY^H\rangle \\&=\langle M, M(Y^HY)^H\rangle\\&=\langle M,M\rangle =\|M\|^2\end{align}$$
Which proves the statement. Does this wok?
I'm afraid your second step $\langle XMY, XMY\rangle=\langle X^HXM,MYY^H\rangle $ is not valid, since we can only do the conversion $X\to X^H$ from the left side, not the right side. The correct proof should be \begin{align}\|XMY\|^2&=\langle XMY, XMY\rangle \\ &=\langle X^HXMY,MY\rangle \\ &=\langle MY,MY\rangle\\ &=\langle Y^HM^H,Y^HM^H\rangle\\&=\langle YY^HM^H,M^H\rangle \\&=\langle M^H,M^H\rangle \\&=\|M\|^2\end{align}