Bessel's correction for standard deviation

596 Views Asked by At

Why does Bessel's correction is biased downward as stated in wiki:

while the sample variance (using Bessel's correction) is an unbiased estimator of the population variance, its square root, the sample standard deviation, is a biased estimate of the population standard deviation; because the square root is a concave function, the bias is downward, by Jensen's inequality.

But by Jensen inequality for a concave function, we have Jensen's inequality for concave functions

where f is the square root function, i.e. the sample standard deviation is greater than the expectation of the srandard deviation, i.e. its biased upward. Where am I wrong?

1

There are 1 best solutions below

0
On BEST ANSWER

You have correctly identified that $f$ is the square root and the convex combination is the integral (expectation).

Therefore, your left hand side $f\bigl( (1-\alpha) x + \alpha y \bigr)$ corresponds to $\sqrt{ E[ S^2 ] }$, where the square root is outside of (taken after) the expectation. Note that $\sqrt{ E[ S^2 ] } = \sqrt{ \sigma^2} = \sigma$ since $S^2$ is unbiased for $\sigma^2$.

At the same time, the right hand side $(1-\alpha)\,f(x) + \alpha\, f(y) $ corresponds to $E\bigl[ \sqrt{S^2} \bigr]$.

Although I don't know where you got that image of Jensen's inequality from, indeed it is correct for the case of concave $f$. This gives us

$$\text{L.H.S.} > \text{R.H.S.} \quad \implies \quad \sigma > E[S]$$

So we see that the $S \equiv \sqrt{S^2}$ is underestimating $\sigma$, while $S^2$ is spot on for $\sigma^2$.

"Where am I wrong?"

What is being pushed upwards by the concavity is $\sigma$, not $S$. In relative terms, one might choose to say that it is $S$ being pushed downward.

The quantity that is bent upward (compared to a straight line) is the one that is "on the curve", which is literally $~f(\cdot)$ and that is $~f\bigl( E[\text{whatever}] \bigr)$ here.