Frobenius norm of product of matrix

Question

The Frobenius norm of a $m \times n$ matrix $F$ is defined as

$$\| F \|_F^2 := \sum_{i=1}^m\sum_{j=1}^n |f_{i,j}|^2$$

If I have $FG$, where $G$ is a $n \times p$ matrix, can we say the following?

$$\| F G \|_F^2 = \|F\|_F^2 \|G\|_F^2$$

Also, what does Frobenius norm mean? Is it analogous to the magnitude of a vector, but for matrix?

Answer 1

Actually there is $$||FG||^2_F \leqslant||F||^2_F||G||^2_F$$ The proof is as follows. \begin{align} \|FG\|^2_F&=\sum\limits_{i=1}^{m}\sum\limits_{j=1}^{p}\left|\sum\limits_{k=1}^nf_{ik}g_{kj}\right|^2 \\ &\leqslant\sum\limits_{i=1}^{m}\sum\limits_{j=1}^{p}\left(\sum\limits_{k=1}^n|f_{ik}|^2\sum\limits_{k=1}^n|g_{kj}|^2\right)\tag{Cauchy-Schwarz} \\ &=\sum\limits_{i=1}^{m}\sum\limits_{j=1}^{p}\left(\sum\limits_{k,l=1}^n|f_{ik}|^2|g_{lj}|^2\right) \\ &=\sum\limits_{i=1}^{m}\sum\limits_{k=1}^{n}|f_{ik}|^2\sum\limits_{l=1}^{n}\sum\limits_{j=1}^{p}|g_{lj}|^2 \\ &=\|F\|^2_F\|G\|^2_F \end{align} Frobenius norm is like vector norm and similar to $l_2$.

Answer 2

In case anyone is curious, there is also a lower bound in a form similar to @passerby51's answer. This result is also used in an ICLR paper, which may be very useful. Let $\mathbf{A}$ be $m \times r$ and $\mathbf{B}$ be $r\times n$. The bounds are $$\sigma_{\min }(\mathbf{A})\|\mathbf{B}\|_{F} \leq \|\mathbf{A B}\|_{F} \leq \sigma_{\max }(\mathbf{A})\|\mathbf{B}\|_{F},$$ where $\sigma_\min$ and $\sigma_\max$ denote the minimum and maximum singular value, respectively.

To prove it, we start with the definition of Frobenius norm, $$ \|\mathbf{A B}\|_{F}^{2}=\operatorname{trace}\left(\mathbf{A B} \mathbf{B}^{\top} \mathbf{A}^{\top}\right)=\operatorname{trace}\left(\mathbf{A}^{\top} \mathbf{A B} \mathbf{B}^{\top}\right) $$ By using the inequalities for matrix trace (see reference below or here), i.e., $ \lambda_{\min }(A) \operatorname{tr}(B) \leq \operatorname{tr}(A B) \leq \lambda_{\max }(A) \operatorname{tr}(B)$, we have $$ \lambda_{\min }\left(\mathbf{A}^{\top} \mathbf{A}\right) \operatorname{trace}\left(\mathbf{B} \mathbf{B}^{\top}\right) \leq \operatorname{trace}\left(\mathbf{A}^{\top} \mathbf{A B} \mathbf{B}^{\top}\right) \leq \lambda_{\max }\left(\mathbf{A}^{\top} \mathbf{A}\right) \operatorname{trace}\left(\mathbf{B} \mathbf{B}^{\top}\right) $$ where $\lambda_\min$ and $\lambda_\max$ denote the minimum and maximum eigenvalues, respectively. This implies, $$ \sigma_{\min }^2(\mathbf{A})\|\mathbf{B}\|_{F}^2 \leq \|\mathbf{A B}\|_{F}^2 \leq \sigma_{\max }^2(\mathbf{A})\|\mathbf{B}\|_{F}^2. $$ Using the fact that all items here are non-negative, we can complete the proof.

Reference:

Y. Fang, et al., Inequalities for the trace of matrix product.