An Estimator Based on Exponential RVs

Question

Let $X_1$, $X_2$, $\cdots$, $X_n$ be $n$ random variables independently sampled from the exponential distribution $\text{exp}(1)$. Suppose $k \leq n$, and $X_{(k)}$ is the $k$-th order statistic, i.e., the $k$-th smallest value in $\{X_1, X_2, \cdots, X_n\}$. Is $\mathbb{E}[\frac{k-1}{X_{(k)}}]$ equal to $n$? If $\text{uniform}(0, 1)$ is used instead of $\text{exp}(1)$, is $\mathbb{E}[\frac{k-1}{X_{(k)}}]$ equal to $n$?

---

What I have obtained. I have known how to compute the pdf and expectation of $X_{(k)}$.

• For the case of $\text{exp}(1)$, we have $$ X_{(k)} = Y_n + Y_{n-1} + \cdots + Y_{n -k + 1}$$ where $Y_i \sim \text{exp}(i)$. Therefore, $$\mathbb{E}(X_{(k)}) = \mathbb{E}(Y_n) + \mathbb{E}(Y_{n-1}) + \cdots + \mathbb{E}(Y_{n-k+1})= \frac{1}{n} + \frac{1}{n-1} + \cdots + \frac{1}{n-k+1}$$ Please refer to http://www.math.kth.se/matstat/gru/sf2955/exponorderstats.pdf

• For the case of $\text{uniform}(0,1)$, we have $$\mathbb{E}(X_{(k)}) = \frac{k}{n+1}$$ Please refer to https://probabilityandstats.wordpress.com/2010/02/21/the-order-statistics-and-the-uniform-distribution/

---

What I have not obtained. I failed to obtain $\mathbb{E}[\frac{k-1}{X_{(k)}}]$ for both $\text{exp}(1)$ and $\text{uniform}(0, 1)$ cases because of the existence of reciprocal. Generally, we do NOT have $$ \mathbb{E}[\frac{1}{X}] = \frac{1}{\mathbb{E}[X]} $$ Any idea to conquer this problem?

---

As suggested by @BruceET's experiments, the answer to the first question is NO, while the answer to the second question is YES. But I need more serious mathematical proof for them.

Answer 1

Look at the generic formula for finding PDFs of order statistics.

In the case of $Exp(1)$ data, the simulation below indicates that $(k-1)/X_{(k)}$ is heavily biased as an estimator for $n,$ for the case $n = 10,\,k = 4.$

 m = 10^6;  n = 10;  k = 4;  x.k = numeric(m)
 for (i in 1:m) {
  x.k[i] = sort(rexp(n))[k] }
 mean((k-1)/x.k)
 ## 8.372766

However, for data from $Unif(0,1)$ it seems that $(k-1)/X_{(k)}$ is unbiased for the case $n = 10,\, k=4.$

 m = 10^6;  n = 10;  k = 4;  x.k = numeric(m)
 for (i in 1:m) {
   x.k[i] = sort(runif(n))[k] }
 mean((k-1)/x.k)
 ## 10.00927
 mean(x.k)
 ## 0.3633717
 k/11
 ## 0.3636364

In the uniform case it should be relatively easy to find the distribution of $X_{k}$. Maybe that is a good place to start. Roughly speaking and on average, the 10 order statistics of a sample of size 10, divide $(0,1)$ into 11 equal parts. That may provide some intuition.

Answer 2

This is a partial answer that I have figured out to my question.

For the $\text{uniform}(0, 1)$ case, we have $$ X_{(k)} \sim \text{beta}(k, \ n - k + 1) $$ Let $Y \sim \text{beta}(n - k + 1, k)$ and $Z \sim \text{beta}^\prime (n - k + 1, k)$, where $\text{beta}^\prime$ denotes the Beta Prime Distribution. Then we have, if $k > 1$, \begin{align} \mathbb{E}[\frac{1}{X_{(k)}}] &= \mathbb{E}[\frac{1}{1 - Y}] = \mathbb{E}[1 + \frac{Y}{1 - Y}] = \mathbb{E}[\frac{Y}{1-Y}] + 1 \\ &= \mathbb{E}[Z] + 1 \\ &= \frac{n-k+1}{k-1} + 1 = \frac{n}{k-1} \end{align} The following facts are used in the equality above.

• If $X \sim \text{beta}(\alpha, \beta)$, then $1 - X \sim \text{beta}(\beta, \alpha)$.
• If $X \sim \text{beta}(\alpha, \beta)$, then $\frac{X}{1 - X} \sim \text{beta}^\prime(\alpha, \beta)$.
• If $X \sim \text{beta}^\prime(\alpha, \beta)$, then $\mathbb{E}[X] = \frac{\alpha}{\beta - 1}$ if $\beta > 1$.

Therefore, we have $$ \mathbb{E}[\frac{k-1}{X_{(k)}}] = n \text{ if } k > 1 $$