Relation between Frobenius norm and trace

Question

Is the following inequality true?

$$\mbox{Tr} \left( \mathrm P \, \mathrm M^T \mathrm M \right) \leq \lambda_{\max}(\mathrm P) \, \|\mathrm M\|_F^2$$

where $\mathrm P$ is a positive definite matrix with appropriate dimension. How about the following?

$$\mbox{Tr}(\mathrm A \mathrm B)\leq \|\mathrm A\|_F \|\mathrm B\|_F$$

Answer 1

The heart of the matter is the following key fact:

The assignment $(A,B) \mapsto \langle A,B \rangle := \operatorname{Tr}(A^T B)$ defines a Euclidean inner product on the real vector space $\mathbb{R}^{m \times n}$ of all $m \times n$ real matrices that induces the Frobenius norm $\|\cdot\|_F$ on $\mathbb{R}^{m\times n}$, in the sense that $$\forall C \in \mathbb{R}^{m \times n}, \quad \|C\|_F = \sqrt{\langle C,C\rangle} = \sqrt{\operatorname{Tr}(C^T C)}.$$

For this, of course, you need the un-normalised trace $\operatorname{Tr}(C) = \sum_{k=1}^n C_{kk}$ on $\mathbb{R}^{n \times n}$.

Once you know this and observe (by whichever definition of the Frobenius norm you prefer) that $\|C^T\|_F = \|C\|_F$ for all $C \in \mathbb{R}^{m \times n}$, the Cauchy–Schwarz inequality for the inner product $\langle \cdot,\cdot \rangle$ immediately yields your second inequality.

The first inequality is a little bit more subtle, but isn't too bad once you recall the following basic fact about traces:

For any orthonormal basis $\{v_1,\dotsc,v_n\}$ of $\mathbb{R}^n$, we have $$\forall C \in \mathbb{R}^{n\times n}, \quad \operatorname{Tr}(C) = \sum_{k=1}^n \langle v_k,Cv_k \rangle.$$

Now, since $P$ is positive definite, let $\{v_1,\dotsc,v_n\}$ be an orthonormal basis for $\mathbb{R}^n$ consisting of eigenvectors of $P$, with $Pv_k = \lambda_k(P)v_k$ for $0 < \lambda_1(P) \leq \cdots \leq \lambda_n(P) = \lambda_{\text{max}}(P)$. If you now write $$ \operatorname{Tr}(PM^T M) = \sum_{k=1}^n \langle v_k,P M^T M v_k \rangle = \langle P v_k,M^T M v_k\rangle = \sum_{k=1}^n \lambda_k(P)\langle v_k, M^T M v_k \rangle $$ in terms of this special orthonormal basis and bound each eigenvalue of $P$ from above by $\lambda_{\text{max}}(P)$, it's now easy to get the first inequality.

Answer 2

Since $\mathrm P$ is positive definite, $\mathrm P^{\frac 12}$ exists and is symmetric. Hence,

$$\mbox{tr} (\mathrm P \mathrm M^T \mathrm M) = \mbox{tr} (\mathrm P^{\frac 12} \mathrm P^{\frac 12} \mathrm M^T \mathrm M) = \mbox{tr} (\mathrm M \mathrm P^{\frac 12} \mathrm P^{\frac 12} \mathrm M^T) = \mbox{tr} ((\mathrm P^{\frac 12} \mathrm M^T)^T (\mathrm P^{\frac 12} \mathrm M^T)) = \|\mathrm P^{\frac 12} \mathrm M^T\|_F^2$$

We have $\|\mathrm A \mathrm B\|_F \leq \|\mathrm A\|_2 \|\mathrm B\|_F$. Thus,

$$\mbox{tr} (\mathrm P \mathrm M^T \mathrm M) = \|\mathrm P^{\frac 12} \mathrm M^T\|_F^2 \leq \|\mathrm P^{\frac 12}\|_2^2 \|\mathrm M^T\|_F^2 = \lambda_{\max} (\mathrm P) \|\mathrm M\|_F^2$$

Answer 3

$$\mbox{tr}(\mathrm A \mathrm B) = \left\langle \mathrm A^\top, \mathrm B \right\rangle = \langle \mbox{vec} (\mathrm A^\top), \mbox{vec} (\mathrm B) \rangle \leq \|\mbox{vec} (\mathrm A^\top)\|_2 \|\mbox{vec} (\mathrm B)\|_2 = \|\mathrm A\|_F \|\mathrm B\|_F$$