Computing $\mathbb{E}[\langle a,c \rangle^4]$ where $a,c \sim \mathcal{N}(0,I_n)$

50 Views Asked by At

I asked a similar question earlier today which can be found here: Expectation of squared inner product. I received a ton of help from @kimchi lover (https://math.stackexchange.com/users/457779/kimchi-lover) and I wanted to share a solution to an extension of that problem.

Let $a,c \sim \mathcal{N}(0,I_n)$ be i.i.d. standard normal gaussian $n$ vectors. We are interested in calculating $$\mathbb{E}[\langle a,c \rangle^4].$$ We will do this by first taking the expectation with respect to $a$ and then with respect to $c$. Fix $c$. Define the random variable $X = \langle a,c \rangle$. Then $X \sim \mathcal{N}(0,\|c\|_2^2)$ since $$\mathbb{E}[X] = \sum_{i=1}^n \mathbb{E}[a_ic_i] = \sum_{i=1}^n c_i\mathbb{E}[a_i] = 0$$ and $$\operatorname{Var}[X] = \mathbb{E}[X^2] = \mathbb{E}[\langle a,c \rangle^2] = \mathbb{E}[c^Taa^Tc] = c^T\mathbb{E}[aa^T]c = c^Tc = \|c\|_2^2$$ where we used the fact that $\mathbb{E}[aa^T] = I_{n \times n}$ (i.e. $a$ is isotropic). Thus $$\mathbb{E}[X^4] = 3 \|c\|_2^4$$ where we used the moment formula: $$\mathbb{E}[Z^{2n}] = \frac{(2n)!}{n!2^n}\sigma^{2n}\ \text{for } Z \sim \mathcal{N}(0,\sigma^2).$$ We now want to find $\mathbb{E}[\|c\|_2^4].$ Note that $\|c\|_2^4$ can be written in the following way:

\begin{align*} \|c\|_2^4 & = \left(c_1^2 + c_2^2 + \dots + c_n^2\right)^2 \\ & = (c_1^4 + c_1^2c_2^2 + \dots + c_1^2c_n^2) + (c_2^2c_1^1 + c_2^4 + \dots + c_2^2c_n^2) + \dots + (c_n^2c_1^2 + c_n^2c_2^2 + \dots + c_n^4) \\ & = \underbrace{\sum_{i_1 = 1}^n c_1^2 c_{i_1}^2}_{ = S_1} + \underbrace{\sum_{i_2 = 1}^n c_2^2 c_{i_2}^2}_{= S_2} + \dots + \underbrace{\sum_{i_n = 1}^n c_n^2 c_{i_n}^2}_{= S_n}. \end{align*} The expectation of each individual sum will be the same. Therefore, we need only consider $S_1$. By linearity of expectation, we get \begin{align*} \mathbb{E}[S_1] & = \mathbb{E}[c_1^4] + \sum_{i_1 \neq 1} \mathbb{E}[c_1^2 c_{i_1}^2] \\ & = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^\infty c_1^4 e^{-c_1^2/2} \, dc_1 + \frac{n-1}{2\pi} \int_{-\infty}^\infty \int_{-\infty}^\infty c_1^2c_{i_1}^2 e^{-(c_1^2+c_{i_1}^2)/2} \, dc_1 \, dc_{i_1}\\ & = 3 + (n - 1) = n+2.\end{align*} Hence $$\mathbb{E}[\|c\|_2^4] = \sum_{i=1}^n \mathbb{E}[S_i] = n(n+2).$$ We conclude that $$\mathbb{E}[\langle a,c \rangle^4] = \mathbb{E}_c\left[\mathbb{E}_a\left[\langle a,c \rangle^4\right]\right] = \mathbb{E}_c \left[3\|c\|_2^4\right] = 3n(n+2).$$

Edit: is there a simple way to calculate the expectation of $\|c\|_2^4$ by using the fact that $\|c\|_2^2$ is a $\chi^2$ random variable with $n$ degrees of freedom?