Expected value of square of dot product of two vectors.

Question

I have a random variable $\epsilon_n$, which is a column vector dependent on $n$, and another vector $w$, which is constant. How do I find the expectation $E[(w^T\epsilon_n)^2]$? It is given that the expected value of $\epsilon_n$ is zero.

My intuition was to write it as $(E[w^T\epsilon_n])^2+Var(w^T\epsilon_n)$, which then becomes $Var(w^T\epsilon_n)$, as the expectation of inner product of a constant vector and another vector is simply the inner product of that constant vector times the expectation of the other(follows from definition of expectation), which in this case is 0. Now I know from the scalar case that $Var(aX)=a^2Var(X)$, where $a$ is a scalar constant. But can I conclude similarly here that $Var(w^T\epsilon_n)=||w||^2Var(\epsilon_n)$?

Answer 1

In the given problem,\begin{align}\mathrm{var}(w^{\top}\epsilon_n)&=w^{\top}\mathrm{cov}(\epsilon_n)w, \end{align} where $\mathrm{cov}(\epsilon_n)=E[\epsilon_n \epsilon_n^{\top}]$, since $\epsilon_n$ is a zero mean random vector.

Answer 2

You cannot make that assumption, for if $w = e_2$, the standard basis vector along the $y$-axis, and $\epsilon_n = U e_1$, where $U$ is uniformly distributed on $[-1, 1]$, then $w \cdot \epsilon_n = 0$, while $\|w\| Var(\epsilon_n) \ne 0$.

On the other hand, if $\epsilon_n = U e_2$, then it works out OK.

In short: that last step requires knowing something more about the relationship between $\epsilon_n$ and $w$.