Proving that the variance of expectation is greater than the variance of sample mean.

Question

We have independent random variables $X_1,\cdots,X_n$, with $\mathbb{E}[X_i]=\mu$ and $Var[X_i]=\sigma^2$ $\forall i$ and the random variable $Z$ defined as

$Z=\sum_{i=1}^{n}c_iX_i$

In the previous part of the question, we are asked to provide a condition on the constant $c_i$ that makes $Z$ and unbiased estimation of $\mu$ which I got as $\sum c_i=1$

The final part which I'm stuck on requires us to show that $Var[Z]\geq Var[\bar{X}]$ where $\bar{X}$ is the sample mean.

My suspicions are that the point of the question is to show that the variance of the expectation is always greater equal to the variance of the sample mean (Although I could be wrong).

I tried starting with the Cauchy Schwarz inequality

$|\sum X_ic_i|\leq \sqrt{\sum X_i^2}\sqrt{\sum c_i^2}$

$Z^2\leq \sum X_i^2\cdot\sum c_i^2$

but cant seem to get anywhere.

Answer 1

Since the $X_i$ are independent, $$ \operatorname{var}{(Z)} = \sum_i \operatorname{var}{(c_i X_i)} = \sigma^2\sum_i c_i^2. $$ Meanwhile, $\operatorname{var}{(\bar{X})} = \sigma^2/n$. But $$ \frac{1}{n} = \frac{1}{n}\left(\sum_i c_i\right)^2 \leq \frac{1}{n}\left( \sum_i 1 \right) \left( \sum_i c_i^2 \right) = \sum_i c_i^2, $$ by Cauchy–Schwarz, and the result follows, with equality iff all $c_i=1/n$.

Answer 2

The inequality $Var[Z] \ge Var[\bar{X}]$ can be reduced to the inequality: $$\sum_{i=1}^{n} c_i^2 \ge \frac{1}{n}.$$ ($c_i$ sum up to 1)

This can be proved using Jensen's inequality (in finite form), taking $f(x) = x^2$, weights $\lambda_i = \frac{1}{n}$ and arguments $x_i = c_i$.