Independence of functions of order statistics when the random variables are uniformly distributed

Question

Let $X_1$,$X_2$,…,$X_n$ be $n$ i.i.d. random variables with $f(x)$ as the pdf and $F(x)$ as the cdf in interval $[0,1]$. Let $F$ be uniformly distributed. Let $X_{i:n}$ be the $i^{th}$ order statistic such that $X_{1:n} \leq X_{2:n} \leq ... \leq X_{n:n}$. I wish to compute the expected value $\mathbb{E} [\frac{X_{(k-1):n} X_{i:n}}{X_{k:n}} ]$ for any $ k < i \leq n$. So the question is are $\frac{X_{(k-1):n}}{X_{k:n}}$ and $X_{i:n}$ independent? Because if they are not, then the problem is non-trivial. Due to a standard result in theory of order statistics, we already know that for any $i \leq n$, $\frac{X_{(i-1):n}}{X_{i:n}}$ and $X_{i:n}$ are independent.

Answer 1

It is easy to show that given $X_{i:n} = x$, the order statistics $X_{1:n}, \dots, X_{(i-1):n}$ have the same joint distribution as the order statistics $X_{1:(i-1)}, \dots, X_{(i-1):(i-1)}$ of a sample from the uniform distribution on $[0,x]$, which, in turn, have the same distribution as $x$ times the order statistics of a sample from $[0,1]$. It follows, in particular, that for $k<i$, $\frac{X_{(k-1):n}}{X_{k:n}}$ is indeed independent of $X_{i:n}$ and $$ \mathrm{E}\Big[\frac{X_{(k-1):n} X_{i:n}}{X_{k:n}} \Big] = \mathrm{E}\Big[\frac{X_{(k-1):n} }{X_{k:n}} \Big]\mathrm{E}[X_{i:n}] = \frac{k-1}k \cdot \frac{i}{n+1}. $$

Answer 2

The following uses (conditional) mutual information from information theory. The only three important property of this that we use are

• Independence : If $X,Y$ are independent random variables and $Z$ is any random variable then $I(X;Y|Z)=0$
• Chain rule : $I(X;Y,Z) = I(X;Y)+I(X;Z|Y)$ (observe the distinction between $,$ and $;$)
• Invariant through one to one processing : if $f:A\rightarrow B$ is a one to one function then $I(X;Y)=I(X;f(Y))$ on properly defined $X$ and $Y$.

Let $Y_{k} = \frac{X_{(k-1)}}{X_{(k)}}$, Suppose that $Y_k$ and $X_{(i)}$ are not independent, then you have $I(Y_k; X_{(i)}) > 0 $ we want to show that $I(Y_k;X_{(n)})>0$. First observe that $Y_{k}$ and $Y_{j}$ for $j\geq i$, then

\begin{align*} I(Y_k;X_{(i)}) &= I(Y_k;X_{(i)}) + \sum_{j=i+1}^n I(Y_k;Y_j|X_{(i)},Y_{i+1},\dots,Y_{j-1})\\ &= I(Y_k;X_{(i)},Y_{i+1},\dots,Y_{n})\\ &= I(Y_k;X_{(n)},Y_{i},\dots,Y_{n-1})\\ &= I(Y_k;X_{(n)}) + \sum_{j=i}^{n-1} I(Y_k;Y_j|X_{(n)},Y_{j+1},\dots,Y_{n-1})\\ &= I(Y_k;X_{(n)}) \end{align*}

This proof can probably be adapted to a non information theoretic one.