Variance of sum of deviations

Question

Suppose I have an i.i.d. sample $\{X_i\}_{i=1}^M$ for some positive integer $M$, and suppose that $X_i \sim X$ for some random variable $X$ with finite variance. Then, denote by $$ E_M = \frac1M\sum_{i=1}^MX_i $$ the empirical mean. The variance of $E_M$ (with respect to the sample) is by independence $$ \mathrm{Var}(E_M)=\mathrm{Var}\left(\frac1M\sum_{i=1}^MX_i\right) = \frac1{M^2}\sum_{i=1}^M\mathrm{Var}\left(X_i\right) = \frac1M \mathrm{Var}(X). $$ All clear and standard. Now consider one point (e.g. $X_1$) and the quantity $E_M - X_1$: $$ E_M - X_1 = \frac1M\sum_{i=1}^MX_i - X_1 = \frac1M\sum_{i=1}^M (X_i - X_1). $$ What about its variance? The increments $X_i - X_1$ are now not independent. Can I still show that $$ \mathrm{Var}(E_M-X_1)\leq \frac{C}M \mathrm{Var}(X), $$ for a constant $C$ independent of $M$?

Answer 1

\begin{align} V(E_M-X_1) &= V\left(\frac{1}{M}\sum_{i=1}^nX_i - X_1 \right) \\ &= V\left(-\frac{M-1}{M}X_1+\frac{1}{M}\sum_{i=2}^nX_i \right) \\ &= \left(\frac{M-1}{M} \right)^2V(X_1)+ \frac{1}{M^2}\sum_{i=2}^nV\left(X \right) \\ &= \left(\left(\frac{M-1}{M} \right)^2 + \frac{M-1}{M^2} \right)V(X) \\ &= \frac{M-1}{M} V(X) \\ \end{align}

You can then deduce easily $C = M-1$ but this $C$ depends on $M$. If $M$ is bounded as $M<b$, for example, then $$C = b-1$$