Let $\left(\mathcal{D}_{n}\right)_{n\in\mathbb{N}}$ be a family of probability distributions such as:
- $\forall n\in\mathbb{N}$, $\mathbb{E}\left[\mathcal{D}_{n}\right]=n$: the average value of the n-th distribution is n.
- $\forall n\in\mathbb{N}$, $\delta\left(\mathcal{D}_{n},\mathcal{D}_{n+1}\right)<\varepsilon$ for some small $\varepsilon>0$, where $\delta$ is the total variation distance: two consecutive distributions are "close to each other".
I want to find a lower bound on $\text{Var}\left(\mathcal{D}_{n}\right)=\mathbb{E}\left[\left(\mathcal{D}_{n}-n\right)^{2}\right]$.
[Edit: I originally conjectured that the bound was something like $\frac{\left(1-\varepsilon\right)}{\varepsilon}\cdot n$, at least asymptotically, but Ross' answer below shows that this is wrong and you can get a family for which each variance is proportional to $1/\varepsilon^2$. I'm looking for any lower bound that is proportional to $1/\varepsilon$ for small values of $\varepsilon$.]
You can have the variance independent of $n$. Let each $\mathcal{D}_{n}$ be a triangular distribution that starts at $n-k$ with value $\varepsilon$, rises in steps of $\varepsilon$ up to $(k+1)\varepsilon$ at $n$, then drops back down to $\varepsilon$ at $n+k$. This set of distributions satisfies the distance requirement. To get total probability of $1$ we need $k+1=\frac 1{\sqrt {\varepsilon}}$. The variance is $$2\sum_{j=1}^k(k-j)\epsilon j^2\approx \frac 1{6\varepsilon}$$ I haven't proved this is the minimum