$E(Z_iZ_j) = E(Z_i)E(Z_j)$ for independent r.v.s

72 Views Asked by At

Consider a population $\omega_1, \omega_2, ..., \omega_N$ of $N$ individuals. Choose, with replacement, a sample of $n$ individuals from the population, denoting this outcome by the ordered list $(\zeta_{1}, \zeta_{2}, ..., \zeta_{N})$. Let $X$ be a random variable. Denote $Z_i = X(\zeta_{i})$ the value of $X$ for the $i^{\text{th}}$ member of the sample.

I am trying to show that the covariance $\text{cov}(Z_i,Z_j)$ of r.v.s $Z_i, Z_j, i \neq j,$ is zero, by showing that $\mathbb{E}(Z_iZ_j) = \mathbb{E}(Z_i)\mathbb{E}(Z_j)$. I get the following $$\text{cov}(Z_i, Z_j) = \mathbb{E}(Z_iZ_j) - \mathbb{E}(Z_i)\mathbb{E}(Z_j),$$ where $\mathbb{E}(Z_i) = \frac{1}{N}(X_1 + \cdots + X_n) = \bar{X}$ and \begin{align} \mathbb{E}(Z_iZ_j) &= \frac{1}{N^2}\sum_{k \neq \ell} x_{k, \ell} \\ &= \frac{1}{N^2} \left(\left(\sum_{i=1}^{N} x_i\right)^2 - \sum_{i=1}^{N} x_i^2 \right) \\ &= \frac{1}{N^2}\left(N^2\bar{X}^2 - \sum_{i=1}^{N} x_i^2\right). \end{align}

It can be. clearly seen from here that my $\text{cov}(Z_i, Z_j) \neq 0$. How can I remedy this?

EDIT: I have emphasised with replacement, thereby implying that $Z_1, Z_2, ..., Z_n$ are independent.

1

There are 1 best solutions below

2
On

Let $X_k = X(\omega_k)$ for $k\in\{1,\dots,N\}$. Let $I_{ik} = \mathbb{1}(\zeta_i = \omega_k) = \mathbb{1}($the $i$th sample is the $k$th individual in the population).

The randomness in this system is your method of choosing $Z_i,Z_j$.

Note that $Z_i = \sum_{k=1}^N X_kI_{ik}$. So

$$ \begin{align} E(Z_iZ_j) &= E\left(\left(\sum_{k=1}^N X_kI_{ik}\right) \cdot\left(\sum_{l=1}^N X_lI_{jl}\right)\right)\\ &=\sum_{k=1}^N\sum_{l=1}^N X_kX_l E(I_{ik}I_{jl})\\ &=\frac{1}{N^2}\sum_{k=1}^N\sum_{l=1}^N X_kX_l \end{align} $$

Note that there's no need to subtract $X_kX_k$: just because $k=l$ doesn't mean we can't have $I_{ik}=I_{jl}=1$; remember we are choosing with replacement.

We also have $E(Z_i) = E\left(\sum_{k=1}^N X_kI_{ik}\right) = \frac1N \sum_{k=1}^N X_k$, so $E(Z_i)E(Z_j) = E(Z_iZ_j)$.