Covariance and Law of Large numbers

Question

Say I am taking the average value of the product of two dependent random variables $X$ and $Y$ sampled an infinite amont of times. That is I am computing $\lim_{n \rightarrow \infty} E \left[ \sum_{i=0}^{n} \frac{Y_{i}X_{i}}{n} \right]$. Is this the same as computing $\lim_{n \rightarrow \infty} E \left[ \sum_{i=0}^{n} \frac{Y_{i}}{n} \right] E \left[ \sum_{i=0}^{n} \frac{X_{i}}{n} \right] = E[X] E[Y]$? Assuming $X$ and $Y$ have finite variance.

I know this would not be true if $n$ was small but does law of large numbers make the covariance $0$ in the same way it makes variance $0$?

Answer 1

From basic properties of expectation, $$E \sum_{i=0}^n \frac{Y_i X_i}{n} = \frac{1}{n} \sum_{i=0}^n E[X_i Y_i] = E[XY]$$ for every $n$. No limits, no law of large numbers.

If you also know $X_i$ and $Y_i$ are uncorrelated (e.g., if they are independent), then $E[X_i Y_i] = E[X_i] E[Y_i]$ and $E[XY] = E[X] E[Y]$.

---

Under the conditions of the law of large numbers (applied to $XY$), we have $$\sum_{i=0}^n \frac{Y_i X_i}{n} \to E[XY]$$ almost surely.

Again, if $X$ and $Y$ are uncorrelated, then $E[XY] = E[X] E[Y]$.

Answer 2

I know this would not be true if n was small but does law of large numbers make the covariance 0 in the same way it makes variance 0?

Let me clarify this. LLN doesn't make the variance of the original random variables 0. It makes the variance of a random variable defined as sample average go to 0.

So $X_n$ would still have the same variance as $X_1$ as n goes to infinity. However if we define, $Z = \left[ \sum_{i=0}^{n} \frac{X_{i}}{n} \right]$, $E[Z] = E[X_i]$ and $Var(Z) = \frac {Var(X)} {n}$. As you see, when n goes to infinity, $Var(Z)$ goes to 0, but Var(X) doesn't change.

Coming back to the original question, $$\lim_{n \rightarrow \infty} E \left[ \sum_{i=0}^{n} \frac{Y_{i}X_{i}}{n} \right] = E[XY]$$

$$E[XY] = E[X]E[Y] + cov(X,Y)$$

Since the $cov(XY)$ isn't going to change no matter the number of n, you cannot calculate the limit the way you proposed in question.