Why $E(y'Ay)\ne E(y)'AE(y)$?

Question

I know that $E(y'Ay)=\mathrm{tr}(A\Sigma) + \mu'A\mu$. However, I am wondering why it is not $E(y')AE(y)$. How can I prove that this is wrong? The explanation given in my textbook is that $E(y'Ay)$ is not linear in $y$. What does it exactly mean?

Answer 1

Let $y_i$ be the $i$-th component of $y$. If $y_i$ and $y_j$ are not independent, then in general $$\mathbb{E}[y_iy_j] \neq \mathbb{E}[y_i]\mathbb{E}[y_j]$$ In other words, in general, $$\mathbb{E}[y^TAy] = \sum_{i, j} A_{ij} \mathbb{E}[y_iy_j] \neq \sum_{i,j} A_{ij} \mathbb{E}[y_i]\mathbb{E}[y_j] = \mathbb{E}[y^T]A\mathbb{E}[y]$$ where we use $$ y^TAy = \sum_{i,j} A_{ij}y_iy_j $$