Why does the jackknife reduce bias

Question

Given a sample $\textbf{x} = (x_1, \ldots, x_n)$, define $\textbf{x}_{(-i)}$ as the sample values excluding sample $x_i$. That is, $$ \textbf{x}_{(-i)} = (x_1, \ldots, x_{i-1}, x_{i+1}, \ldots x_n). $$ Now given estimator $T(\textbf{x})$ of parameter $\tau(\theta)$ based on the sample $\textbf{x}$, Lehmann and Casella (in the 2nd edition of Theory of Point Estimation) define the jackknifed version of $T(\textbf{x})$ as $$ T_J(\textbf{x}) = nT(\textbf{x}) - \frac{n -1}{n}\sum_{i=1}^n T\left(\textbf{x}_{(-i)}\right). $$ They claim that if $E[T(\textbf{x})] = \tau(\theta) + O\left(\frac{1}{n}\right)$, then $E[T_J(\textbf{x})] = \tau(\theta) + O\left(\frac{1}{n^2}\right)$. Can someone explain why the jackknifed version of $T(\textbf{x})$ has lower asymptotic bias then the original estimator?

Answer 1

I figured this out, but it requires a constraint slightly stronger than the original problem statement. In the original statement, they claim the result is true if $E[T(\textbf{x})] = \tau(\theta) + O\left(\frac{1}{n}\right)$. I was getting hung up on pathological cases, like: $$ E[T(\textbf{x})] = \tau(\theta) + \frac{1}{n} + \frac{1}{n^{1.5}}, $$ and I don't believe the result is actually true in such cases. If you assume the slightly stronger constraint $$ E[T(\textbf{x})] = \tau(\theta) + \sum_{i=1}^\infty\frac{b_i}{n^i}, $$ the result becomes provable. Note that in many practical cases, the estimator will be representable in this constrained form of $O\left(\frac{1}{n}\right)$ functions, so this extra constraint does not dillute the power of the jackknife method in reducing bias. Here's the proof: $$ \begin{align} E[T_J(\textbf{x})] &= E\left[nT(\textbf{x}) - \frac{n -1}{n}\sum_{j=1}^n T\left(\textbf{x}_{(-j)}\right)\right]\\ &= nE[T(\textbf{x})] - \frac{n -1}{n}\sum_{j=1}^n E\left[T\left(\textbf{x}_{(-j)}\right)\right] \\ &= n\left(\tau(\theta) + \sum_{i=1}^\infty\frac{b_i}{n^i} \right) - \frac{n -1}{n}\left(\sum_{j=1}^n \left(\tau(\theta) + \sum_{i=1}^\infty \frac{b_i}{(n-1)^i}\right)\right) \\ &=n\tau(\theta) + \sum_{i=1}^\infty\frac{b_i}{n^{i-1}} - (n-1)\left(\tau(\theta) + \sum_{i=1}^\infty \frac{b_i}{(n-1)^i}\right) \\ &= \tau(\theta) + \sum_{i=1}^\infty \left(\frac{b_i}{n^{i-1}} - \frac{b_i}{(n-1)^{i-1}}\right) \\ &= \tau(\theta) + \left(b_1 - b_1\right) + \sum_{i=2}^\infty \left(\frac{b_i}{n^{i-1}} - \frac{b_i}{(n-1)^{i-1}}\right) \\ &= \tau(\theta)+ \sum_{i=2}^\infty b_i\frac{(n-1)^{i-1} - n^{i-1}}{n^{i-1}(n-1)^{i-1}} \end{align} $$ Letting $$ c_i = \frac{(n-1)^{i-1} - n^{i-1}}{n^{i-1}(n-1)^{i-1}}, $$ it's not hard to see from a binomial expansion in the numerator that $c_i = O\left(\frac{1}{n^i}\right)$, so $$ E[T_J(\textbf{x})] = \tau(\theta) + O\left(\frac{1}{n^2}\right) $$