$\operatorname{Var}(x^TAx) = 2\|A\|_F^2$ can't find mistake

Question

Let $x \in \mathbb R^n$ be a vector of random iid numbers, drawn from normal distribution with mean zero and variance 1, and let $A$ be symmetric $n$ by $n$ real matrix.

I want to show that $\operatorname{Var}(x^TAx) = 2\|A\|_F^2$

What I tried:

First, I'll mention that I already proved that $E[x^TAx] = Tr(A)$. I'll use it here.

$\operatorname{Var}[x^TAx] = E[(x^TAx)^2] - (E[x^TAx])^2 = E[x^TAxx^TAx] - (Tr(A))^2$

Since $A$ is symmetric, there's an orthonormal matrix $U$ and diagonal matrix $D$ such that $A = U^TDU$:

$E[x^TAxx^TAx] - (\operatorname{Tr}(A))^2 = E[x^TU^TDUxx^TU^TDUx] - (\operatorname{Tr}(A))^2 = \\ E[y^TDyy^TDy] - (\operatorname{Tr}(A))^2$

Where $y := Ux$, which is also iid random variables with zero mean and 1 variance.

$\displaystyle E[y^TDyy^TDy] - (\operatorname{Tr}(A))^2 = E[\sum_{i=1}^{n}\lambda_iy_i^2 \cdot\sum_{j=1}^{n}\lambda_j y_j^2] - (\operatorname{Tr}(A))^2 = \\ \displaystyle \sum_{i=1}^{n}\sum_{j=1}^{n}\lambda_i\lambda_jE[y_i^2y_j^2] - (\operatorname{Tr}(A))^2$

Since $y_i, y_j$ are iid, $E[y_i^2 y_j^2 ] = E[y_i^2]E[y_j^2] = 1$ and so we finally have

$\displaystyle \sum_{i=1}^{n}\sum_{j=1}^{n}\lambda_i\lambda_j - (\operatorname{Tr}(A))^2$

Now maybe I'm crazy, but isn't this zero?

Answer 1

The error is assuming that $y_i$ and $y_j$ are iid for all $i$ and $j$.

This is true if $i\ne j$, but you should consider separately the case where $i=j$.

Answer 2

Starting from $$\sum_i \sum_j \lambda_i \lambda_j E[y_i^2 y_j^2] - \text{Tr}(A)^2.$$ Split into cases $i=j$ and $i\neq j$. This results in the sum breaking into two parts $$\sum_{i\neq j} \lambda_i \lambda_j + 3 \sum_i \lambda_i^2 - \text{Tr}(A)^2.$$ Note that we have used the identity the $E[y_i^4] = 3$ in the above formula. Now, $$\sum_{i\neq j} \lambda_i \lambda_j = \sum_i \sum_j \lambda_i \lambda_j - \sum_i \lambda_i^2 = \left(\sum_i \lambda_i \right)^2 - \sum_i \lambda_i^2. $$ Plugging this into the equation gives $$2 \sum_i \lambda_i^2 = 2 \| A \|_F.$$

Answer 3

Others have identified the error, so let's do a concise corrected calculation. Since $E[x_ix_j]=\delta_{ij}$ and $E[x_ix_jx_kx_l]=\delta_{ij}\delta_{kl}+\delta_{ik}\delta_{jl}+\delta_{il}\delta_{jk}$,$$\begin{align}\operatorname{Var}(A_{ij}x_ix_j)&=A_{ij}A_{kl}(\delta_{ij}\delta_{kl}+\delta_{ik}\delta_{jl}+\delta_{il}\delta_{jk})-(A_{ij}\delta_{ij})^2\\&=A_{ij}A_{kl}(\delta_{ik}\delta_{jl}+\delta_{il}\delta_{jk})\\&=A_{ij}A_{ij}+A_{ij}A_{ji}.\end{align}$$By symmetry, this simplifies to $2A_{ij}A_{ij}=2\Vert A\Vert_F^2$.