From $E[XY]$ deriving $E[X^2]$ doesn't match up!

Question

$$E[XY] = \sum_{x\epsilon X }^{} \sum_{y\epsilon Y}^{} xy \cdot P(xy)$$

so if we let $Y=X$ we get

$$E[X^2] = \sum_{x \in X }^{} \sum_{y \in X}^{} xy *P(xy)$$

but $$E[X^2] = \sum_{x \in X}^{}x^2P(x)$$,

if I try these to equations with a simple example like X = {1, 2, 3} each with probability 1/3, I get different answers.

Can someone please let me know where I went wrong in my logic ?

Also, let's say X has 5 elements and Y has 5 elements, would XY have 25 elements? If so, why doesn't XX have 25 elements, and is only understood to mean each value of an element is simply squared?

Answer 1

You replace $x\cdot y$ by $x\cdot x=x^2$ only. $$E[X^2] = \sum_{x\epsilon X }^{} \sum_{y\epsilon Y}^{} x^2 \cdot P(x,y)$$

$$= \sum_{x\epsilon X }^{} x^2 \cdot \sum_{y\epsilon Y}^{} P(x,y)= \sum_{x\epsilon X }^{} x^2 \cdot P(x), $$

where $P(x)$ are the corresponding probabilities of $X$.

Answer 2

Let's go back to a simple numeric example to illustrate the difference.

Let $X$ be the outcome of a fair six-sided die that is colored green; i.e., $$\Pr[X = x] = \frac{1}{6}, \quad x \in \{1, 2, 3, 4, 5, 6\}.$$ Similarly, let $Y$ be the outcome of a second fair six-sided die, which is colored red. You toss both dice and observe the outcome. What is the average of the product of the outcome? For example, you could have $X = 2$, $Y = 3$, and $XY = 6$; we want $\operatorname{E}[XY]$. This is simply $$\begin{align} \operatorname{E}[XY] &= \sum_{x=1}^6 \sum_{y=1}^6 xy \Pr[X = x, Y = y] \\ &\overset{\text{ind}}{=} \sum_{x=1}^6 \sum_{y=1}^6 xy \Pr[X = x]\Pr[Y = y] \\ &= \sum_{x=1}^6 x \Pr[X = x] \sum_{y=1}^6 y \Pr[Y = y] \\ &= \operatorname{E}[X]\operatorname{E}[Y] \\ &= \frac{7}{2} \cdot \frac{7}{2} \\ &= \frac{49}{4}. \end{align}$$

Now, suppose instead we want the average value of the square of the green die only, ignoring the red die's outcome completely. This is the meaning of $\operatorname{E}[X^2]$. The set of outcomes for the square of the green die's value is $X^2 \in \{1, 4, 9, 16, 25, 36\}$, since $X \in \{1, 2, 3, 4, 5, 6\}$. So we must have $$\operatorname{E}[X^2] = \sum_{x=1}^6 x^2 \Pr[X = x] = \frac{1 + 4 + 9 + 16 + 25 + 36}{6} = \frac{91}{6}.$$

This shows how the two expectations are not the same. It would have been the same if, for instance, the red die was somehow magically linked to the green one, and always showed the same number as the green. But it is not, so you could have gotten $XY = 6$ but this outcome is impossible for $X^2$, as $6$ is not a perfect square.

When you have the product of two random variables, say $X$ and $Y$, the outcome of one variable may not be completely determined by the outcome of the other. We can see from the dice example that $XY = X^2$ only if the variable $Y$ is perfectly correlated and equal to $X$. It is not sufficient that $Y$ simply obeys the same distribution as $X$ does: its outcome must be identical to the outcome of $X$.