Expectation of sample standard deviation

Question

Suppose we draw a random sample $X_1,X_2,\ldots,X_n$ from $N(\mu,\sigma^2)$ population. Let $S^2$ be the sample variance given by $\frac{1}{n-1}\sum_{i=1}^{n}(X_i-\bar{X})^2$ Now, we are to find $E(S)$.

It is simple enough if we find the pdf of $S$ by transformations and then calculate $E(S)$ by standard method. But is there a simpler and less tedious approach?

Answer 1

$\newcommand{\v}{\operatorname{var}}\newcommand{\c}{\operatorname{cov}}\newcommand{\e}{\operatorname{E}}$One way is this: Find \begin{align} & \e \big( (X_i-\overline X\,)^2\big) =\v(X_i-\overline X\,) = \v(X_i) + \v(\overline X\,) - 2\c(X_i,\overline X\,) \\[10pt] = {} & \sigma^2 + \frac{\sigma^2} n - 2\frac{\sigma^2} n \end{align} and then add those up.

PS: It is pointed out that $\e S$ rather than $\e(S^2)$ was required. That's more involved. How to show that $$ Y = (n-1) \frac{S^2}{\sigma^2} \sim \chi^2_{n-1} $$ is a question that I think has been dealt with in these pages. So we have $$ f_Y(y) = \frac 1 {\Gamma(n/2)} \left( \frac y 2 \right)^{(n/2)-1} e^{-y/2} \, \frac{dy} 2 \quad\text{for } y\ge0. $$ And then \begin{align} \e S & = \frac \sigma {\sqrt{n-1}} \int_0^\infty \sqrt y \, f_Y(y)\, dy \\[10pt] & = \frac {\sigma\sqrt 2}{\sqrt{n-1}} \cdot \frac 1 {\Gamma(n/2)} \int_0^\infty \sqrt{\frac y 2} \left( \frac y 2 \right)^{(n/2)-1} e^{-y/2} \, \frac{dy} 2 \\[10pt] & = \frac {\sigma\sqrt 2}{\sqrt{n-1}} \cdot \frac 1 {\Gamma(n/2)} \int_0^\infty u^{((n+1)/2)-1} e^{-u} \, du \\[10pt] & = \frac {\sigma\sqrt 2}{\sqrt{n-1}} \cdot \frac 1 {\Gamma(n/2)} \cdot \Gamma\left( \frac{n+1} 2 \right) \end{align} Now iteratively apply the identity $\Gamma(\alpha+1) = \alpha \Gamma(\alpha)$ and remember that $\Gamma(1/2) = \sqrt \pi.$

Answer 2

Let $\iota=(1,\ldots,1)'$ (the $n\times 1$ vector with entries all $1$). Let $X=(X_1,\ldots,X_n)$ and $M=I_n-\iota(\iota'\iota)^{-1}\iota'$. It's easy to verify that $M$ has rank $n-1$ so $$ \frac{n-1}{\sigma^2}S^2=\frac{1}{\sigma^2}X'MX=[(X-\mu\iota)/\sigma]'M[(X-\mu\iota)/\sigma]\sim\chi^2_{n-1} $$ the $\chi^2$ distribution with $n-1$ degrees of freedom. So $\sqrt{\frac{n-1}{\sigma^2}S^2}$ has the $\chi$ distribution with $n-1$ degrees of freedom. It follows that $$ E(S)=\frac{\sigma}{\sqrt{n-1}}E\left[\sqrt{\frac{n-1}{\sigma^2}S^2}\right]=\frac{\sqrt{2}\sigma}{\sqrt{n-1}}\frac{\Gamma(n/2)}{\Gamma((n-1)/2)}. $$ Note that while $S^2$ is unbiased for $\sigma^2$ (as indicated by some other answers in this thread or those given here), $S$ is not unbiased for $\sigma$. Indeed, you can plot $\frac{\sqrt{2}}{\sqrt{n-1}}\frac{\Gamma(n/2)}{\Gamma((n-1)/2)}$:

As you see, $\frac{\sqrt{2}}{\sqrt{n-1}}\frac{\Gamma(n/2)}{\Gamma((n-1)/2)}$ approaches $1$ as $n\to\infty$ but it is nonetheless not $1$.