Frobenius norm of product of matrix and vector

Question

I need to find the condition for $\|Ax\|_F^2$=$\|A\|_F^2$$\|x\|_F^2$, where A is a matrix and x is a vector. Can anyone help me? Thank you.

Answer 1

The Frobenius norm is unitarily invariant, i.e. $\|U A V\|_F = \|A\|_F$, for any $U$ and $V$ unitary. So we may assume that $A = \text{diag}(d_1, d_2, \cdots, d_n)$ is diagonal. (The left hand side can be rewritten as $\|U D V^\dagger (V y)\|_F$, where $UDV^\dagger$ is the singular value decomposition of $A$ and $y = V^\dagger x$). Letting $y = (y_1, y_2, \cdots, y_n)^T$, we have $$\|UDV (Vy)\|_F^2 = \|D y\|_F^2 = \sum_{i = 1}^n |d_i y_i|^2$$ We want this value to equal $\|A\|_F^2 \|x\|_F^2 = \|D\|_F^2 \|y\|_F^2$, namely we want $$\sum_{i = 1}^n |d_i y_i|^2 = \left(\sum_{i = 1}^n |d_i|^2 \right) \left(\sum_{i = 1}^n |y_i|^2 \right)$$ For equality to hold we must have $$\sum_{i \neq j} |d_i|^2 |y_j|^2 = 0$$ As all terms in the summation are non-negative, we must have each term be zero. So ultimately, we can either have $D = 0$ (implying $A = 0$) or $y = 0$, or have exactly one index $i$ such that $d_i, y_i \neq 0$. So in summary, the equality holds if and only if one of the following is true:

1. $A = 0$
2. $y = 0$
3. $A$ is a rank one matrix (i.e. its singular values are all zero except one, say $d_1$) and $y_1$ is a multiple of the right-singular vector corresponding to the nonzero singular value. An example is $$A = \begin{bmatrix} 1 & 0 & \cdots & 0 \\ 0 & 0 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 0\end{bmatrix}, ~~ x = \begin{bmatrix} 1 \\ 0 \\ \vdots \\ 0\end{bmatrix}$$

We can also conclude from our steps that in all other cases, we have $\|A x\|_F^2 < \|A \|_F^2 \|x\|_F^2$ (the Frobenius norm is sub-multiplicative).

Addendum: Why must $y$ be a right-singular vector?

Answer: If we order the singular values (all of which are zero except one) of $A$ in descending order, i.e. $D = \text{diag}(d_1, \cdots, d_n)$ with $d_1 > 0$ and $d_2 = d_3 = \cdots = d_n = 0$, then we want $V^\dagger y = x = \hat{e}_1$ (here $\hat{e}_1$ is the first standard basis vector). Thus $y = V \hat{e}_1$ (which picks out the first column of $V$). This is the definition of the right-singular vector corresponding to the nonzero singular value (the columns of $V$ are the right-singular vectors of $A$).