Gaussian expectation calculation

Question

If $x$ is a fixed unit vector and $a \sim \mathcal{N}(0, I)$, then

$$\mathbb{E}\left[ (a^{\top}x)^{2}aa^{\top}\right] = 2xx^{\top} + I$$

This can be intuitively verified when $x$ is a unit vector corresponding to coordinate axes $x = \mathrm{e}_i$ by using fourth moment of gaussian $\mathbb{E}\left[a_i^4\right] = 3$ and pairwise independence between the components $a_i$ of $a$.

However I am not able to complete the proof of above moment calculation.

\begin{align*} \mathbb{E}\left[ (a^{\top}x)^{2}aa^{\top}\right] &= \mathbb{E}\left[ a^{\top}xx^{\top}aaa^{\top}\right] \\ &= \mathbb{E}\left[ x^{\top}aa^{\top}xaa^{\top}\right] \\ &= \mathbb{E}\left[ x^{\top}\left(aa^{\top}-I\right)xaa^{\top}\right] + I \\ \end{align*}

Answer 1

I couldn't find a nice trick, but computing each entry directly seems to work.

The $(i,j)$ entry of $aa^\top$ is $a_i a_j$, so the $(i,j)$ entry of $(a^\top x)^2 aa^\top$ is $a_i a_j \left(\sum_k a_k x_k\right)^2 = a_i a_j \sum_{k,l} a_k a_l x_k x_l$ .

• If $i=j$, then the expectation is $\sum_{k \ne i} x_k^2 + 3 x_i^2 = \|x\|^2 + 2 x_i^2 = 1 + 2 x_i^2$, because the only terms that survive are $k=l$; you need to handle the cases $i=j=k=l$ and $i=j \ne k=l$ separately though.
• If $i \ne j$, then the expectation is $2 x_i x_j$ because the only two addends that survive are $k=i,l=j$ and $k=j, l=i$.

So, the expectation of the whole matrix is $2xx^\top + I$.

Answer 2

Let $A = (a^Tx)^2aa^T$, then,

$$A_{ij} = a_ia_j \sum_{p=1}^{n}\sum_{q=1}^{n}a_pa_qx_px_qa_ia_j =\sum_{p=1}^{n}\sum_{q=1}^{n}a_ia_ja_pa_qx_px_q$$

$$E(A_{ij}) = \sum_{p=1}^{n}\sum_{q=1}^{n}E(a_ia_ja_pa_q)x_px_q$$

Cases:

1. When $i \neq j$, then the term $E(a_ia_ja_pa_q)$ in above expression will be non zero when $p=i, q=j$ or $p=j, q=i$. Therefore,

$$E(A_{ij}) = 2x_ix_j$$

2. When $i \neq j$, then the term $E(a_ia_ja_pa_q)$ in above expression will be non zero when $p=q=i$ or $p = q \neq i$. Therefore,

$$E(A_{ii}) = \sum_{k=1, k \neq i}^{n}x_kx_k + 3x_i^2 = \sum_{k=1}^{n}x_k^2 + 2x_i^2 = 1 + 2x_i^2$$