Frobenius norm : How do we get rid of the matrices $V$ and $V^T$?

Question

I want to show that the Frobenius norm is unitary invariant, i.e. I want to show that $\|UAV\|_F=\|A\|_F$ with $U,V\in \mathbb{R}^{n\times n}$ unitary matrices.

First I have shown that $\|A\|_F=\left (\text{trace}(A^TA)\right )^{1/2}$ as follows :

We have that \begin{equation*}[AB](k,j) = \sum_{i=1}^n a_{ki}b_{ij} \ \text{ so } \ \text{trace}(AB) = \sum_{k,j = 1}^n a_{kj}b_{jk}\end{equation*} So we get \begin{equation*}\text{trace}(A^TA) = \sum_{k,j = 1}^n a_{kj}^Ta_{jk}=\sum_{k,j = 1}^n a_{jk}a_{jk}=\sum_{k,j = 1}^n a_{jk}^2=\sum_{j,k=1}^n|a_{j,k}|^2=\|A\|_F^2\end{equation*}

Let $U,V\in \mathbb{R}^{n\times n}$ be unitary matrices.

We have that \begin{align*}\|UAV\|_F^2&=\text{trace}\left [\left (UAV\right )^T\left (UAV\right )\right ]=\text{trace}\left [\left (V^TA^TU^T\right )\left (UAV\right )\right ]\\ & =\text{trace}\left [V^TA^TU^TUAV\right ]=\text{trace}\left [V^TA^T(U^TU)AV\right ]\\ & =\text{trace}\left [V^TA^TIAV\right ]=\text{trace}\left [V^T(A^TA)V\right ]\end{align*} How dan we get the product $V^TV$ since the matrix $A^TA$ is between them?

Answer 1

There is at least 3 different ways to approach this problem.

1. Single-sided orthogonal transformations preserve the Frobenius norm.
2. The trace identity $Tr(AB) = Tr(BA)$
3. Similarity transforms preserve the characteristic polynomial and therefore also the trace.

---

Approach 1. If $U$ is an orthogonal square matrix and $x$ is a vector such that $y=Ux$ is defined, then $$\|y\|_2^2 = (Ux)^TUx = x^TU^TUx = x^Tx = \|x\|_2^2.$$ By focusing on the columns of $A = \begin{bmatrix} c_1 & c_2 & \dotsc & c_m\end{bmatrix}$ we see that $$\|UA\|_F^2 = \sum_{j=1}^m \|Uc_j\|_2^2 =\sum_{j=1}^m \|c_j\|_2^2 = \|A\|_F^2.$$ Now, let $V$ be orthogonal and set $B = UA$. Then $$\|UAV^T\|_F^2 = \|BV^T\|_F = \|V B^T\|_F = \|B\|_F = \|A\|_F,$$ because $\|X\|_F = \|X^T\|$. This completes the proof.

---

Approach 2. Let $A$ be $m$-by-$k$ and let $B$ be $k$-by-$m$, so that $AB$ and $BA$ are defined. Then $$\text{tr}(AB) = \sum_{i=1}^m (AB)_{ii} = \sum_{i=1}^m \sum_{j=1}^k a_{ij}b_{ji} = \sum_{j=1}^k \sum_{i=1}^m b_{ji} a_{ij} = \sum_{j=1}^k (BA)_{jj} = \text{tr}(BA).$$ It follows that if $V$ is square and orthogonal, then $$\text{tr}(V^T(A^TAV)) = \text{tr}((A^TAV)V^T) = \text{tr}((A^TA)(VV^T)) = \text{tr}(A^TA).$$

---

Approach 3. This approach uses the fact that the characteristic polynomial of matrix, i.e., the polynomial $$p_A(t) = \det(A-tI)$$ satisfies $$p_A(t) = (-t)^m + \text{tr}(A)(-t)^{m-1} + \dotsc + \det(A).$$ Moreover, the characteristic polynomial is preserved by similarity transforms. If $X$ is nonsingular and $B = XAX^{-1}$, then $A$ and $B$ have the same trace, because they have the same characteristic polynomial. If $V$ is orthogonal, then $A^TA$ and $V^T(A^TA)V$ are similar, hence they have the same trace.

---

The approach taken should reflect the audience.