Joint distribution of order statistics proof

Question

I want to rigorously derive the density function of $(X_{(i)},X_{(j)})$ for an order statistics of a random sample $X_1,...,X_n$ from a continuous population with cdf $F(x)$ and pdf $f_X(x)$. I am aware of it's form as given in wikipedia. However,I am unable to provide a rigorous argument for the same .

Answer 1

The joint CDF of $X_{(i)},X_{(j)}$ is given by $F_{X_{(i)},X_{(j)}}(u,v) = \sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k}\frac{n!}{k!m!(n-k-m)!}[F_{X}(u)]^k[F_{x}(v)-F_{x}(u)]^m[1-F_{X}(v)]^{n-k-m} + P(U \geq j)$. The argument for this is explained in Statistical Inference by Casella and Berger problem 5.26 pretty clearly. I can elaborate more on it if I need to. Deriving the PDF is the hard part. This is my solution, it looks longer than it really is. Recall that to find the joint PDF we take the mixed partial derivative: $$f_{X_{(i)},X_{(j)}}(u,v)=\frac{\partial}{\partial u \partial v}F_{X_{(i)},X_{(j)}}(u,v)$$ Passing the partial through the double summation (the mixed partial of $P(U \geq j)$ is $0$) and applying the chain rule we get the following $$\frac{\partial}{\partial u \partial v}F_{X_{(i)},X_{(j)}}(u,v) = \\ \sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k} \frac{n!}{k!m!(n-k-m)!}mkf(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m} \\ -\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k} \frac{n!}{k!m!(n-k-m)!}m(m-1)f(v)f(u)[F_{X}(u)]^{k}[F_{X}(v)-F_{X}(u)]^{m-2}[1-F_{X}(v)]^{n-k-m} \\ -\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k} \frac{n!}{k!m!(n-k-m)!}k(n-k-m)f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m}[1-F_{X}(v)]^{n-k-m-1}\\ +\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k} \frac{n!}{k!m!(n-k-m)!}m(n-k-m)f(v)f(u)[F_{X}(u)]^{k}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m-1}$$ Now, notice that for the first term $$\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k} \frac{n!}{k!m!(n-k-m)!}mkf(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m} = \\ \frac{n!}{(i-1)!(j-i-1)!(n-j)!}f(v)f(u)[F_{X}(u)]^{i-1}[F_{X}(v)-F_{X}(u)]^{j-i-1}[1-F_{X}(v)]^{n-j} \\ + \sum\limits_{k=i+1}^{j-1}\sum\limits_{m=j-k}^{n-k} \frac{n!}{k!m!(n-k-m)!}mkf(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m} \\ + \sum\limits_{k=i}^{i}\sum\limits_{m=j-k+1}^{n-k} \frac{n!}{k!m!(n-k-m)!}mkf(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m} $$ So we need to show that the following is equal to zero: $$\sum\limits_{k=i+1}^{j-1}\sum\limits_{m=j-k}^{n-k} \frac{n!}{k!m!(n-k-m)!}mkf(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m} \\ + \sum\limits_{k=i}^{i}\sum\limits_{m=j-k+1}^{n-k} \frac{n!}{k!m!(n-k-m)!}mkf(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m} \\ -\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k} \frac{n!}{k!m!(n-k-m)!}m(m-1)f(v)f(u)[F_{X}(u)]^{k}[F_{X}(v)-F_{X}(u)]^{m-2}[1-F_{X}(v)]^{n-k-m} \\ -\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k} \frac{n!}{k!m!(n-k-m)!}k(n-k-m)f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m}[1-F_{X}(v)]^{n-k-m-1}\\ +\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k} \frac{n!}{k!m!(n-k-m)!}m(n-k-m)f(v)f(u)[F_{X}(u)]^{k}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m-1}$$ Okay so notice that the two bottom terms are equal to zero for $m = n- k$ so we can rewrite this as $$\sum\limits_{k=i+1}^{j-1}\sum\limits_{m=j-k}^{n-k} \frac{n!}{k!m!(n-k-m)!}mkf(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m} \\ + \sum\limits_{k=i}^{i}\sum\limits_{m=j-k+1}^{n-k} \frac{n!}{k!m!(n-k-m)!}mkf(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m} \\ -\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k} \frac{n!}{k!m!(n-k-m)!}m(m-1)f(v)f(u)[F_{X}(u)]^{k}[F_{X}(v)-F_{X}(u)]^{m-2}[1-F_{X}(v)]^{n-k-m} \\ -\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k-1} \frac{n!}{k!m!(n-k-m)!}k(n-k-m)f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m}[1-F_{X}(v)]^{n-k-m-1}\\ +\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k-1} \frac{n!}{k!m!(n-k-m)!}m(n-k-m)f(v)f(u)[F_{X}(u)]^{k}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m-1}$$ First lets look at the third and fifth summations. $$-\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k} \frac{n!}{k!m!(n-k-m)!}m(m-1)f(v)f(u)[F_{X}(u)]^{k}[F_{X}(v)-F_{X}(u)]^{m-2}[1-F_{X}(v)]^{n-k-m} \\ \sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k-1} \frac{n!}{k!m!(n-k-m)!}m(n-k-m)f(v)f(u)[F_{X}(u)]^{k}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m-1} = \\ -\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k} \frac{n!}{k!(m-2)!(n-k-m)!}f(v)f(u)[F_{X}(u)]^{k}[F_{X}(v)-F_{X}(u)]^{m-2}[1-F_{X}(v)]^{n-k-m} \\ \sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k-1} \frac{n!}{k!(m-1)!(n-k-m-1)!}f(v)f(u)[F_{X}(u)]^{k}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m-1}$$ Change the dummy index on the third summation from $m$ to $m+1$ and get $$-\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k-1}^{n-k-1} \frac{n!}{k!(m-1)!(n-k-m-1)!}f(v)f(u)[F_{X}(u)]^{k}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m-1} \\ \sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k-1} \frac{n!}{k!(m-1)!(n-k-m-1)!}f(v)f(u)[F_{X}(u)]^{k}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m-1} \\ =-\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k-1}^{j-k-1} \frac{n!}{k!m!(n-k-m)!}m(n-k-m)f(v)f(u)[F_{X}(u)]^{k}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m-1} $$ Let's put that result aside and look at the first and fourth summation $$\sum\limits_{k=i+1}^{j-1}\sum\limits_{m=j-k}^{n-k} \frac{n!}{k!m!(n-k-m)!}mkf(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m} \\ -\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k-1} \frac{n!}{k!m!(n-k-m)!}k(n-k-m)f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m}[1-F_{X}(v)]^{n-k-m-1} \\ = \sum\limits_{k=i+1}^{j-1}\sum\limits_{m=j-k}^{n-k} \frac{n!}{(k-1)!(m-1)!(n-k-m)!}f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m} \\ -\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k-1} \frac{n!}{(k-1)!m!(n-k-m-1)!}f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m}[1-F_{X}(v)]^{n-k-m-1}$$ Changing the dummy index on the first summation from $m$ to $m+1$ we have $$\sum\limits_{k=i+1}^{j-1}\sum\limits_{m=j-k}^{n-k} \frac{n!}{(k-1)!(m-1)!(n-k-m)!}f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m} \\ -\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k-1} \frac{n!}{(k-1)!m!(n-k-m-1)!}f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m}[1-F_{X}(v)]^{n-k-m-1} \\ = \sum\limits_{k=i+1}^{j-1}\sum\limits_{m=j-k-1}^{n-k-1} \frac{n!}{(k-1)!m!(n-k-m-1)!}f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m}[1-F_{X}(v)]^{n-k-m-1} \\ -\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k}^{n-k-1} \frac{n!}{(k-1)!m!(n-k-m-1)!}f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m}[1-F_{X}(v)]^{n-k-m-1} \\ = -\sum\limits_{k=i}^{i}\sum\limits_{m=j-k}^{n-k-1} \frac{n!}{k!m!(n-k-m)!}k(n-k-m)f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m}[1-F_{X}(v)]^{n-k-m-1} \\ + \sum\limits_{k=i+1}^{j-1}\sum\limits_{m=j-k-1}^{j-k-1} \frac{n!}{k!m!(n-k-m)!}k(n-k-m)f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m}[1-F_{X}(v)]^{n-k-m-1} $$ Putting these results together, we must show that the following is zero: $$-\sum\limits_{k=i}^{i}\sum\limits_{m=j-k}^{n-k-1} \frac{n!}{k!m!(n-k-m)!}k(n-k-m)f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m}[1-F_{X}(v)]^{n-k-m-1} \\ + \sum\limits_{k=i+1}^{j-1}\sum\limits_{m=j-k-1}^{j-k-1} \frac{n!}{k!m!(n-k-m)!}k(n-k-m)f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m}[1-F_{X}(v)]^{n-k-m-1}\\ -\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k-1}^{j-k-1} \frac{n!}{k!m!(n-k-m)!}m(n-k-m)f(v)f(u)[F_{X}(u)]^{k}[F_{X}(u)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m-1} \\ + \sum\limits_{k=i}^{i}\sum\limits_{m=j-k+1}^{n-k} \frac{n!}{k!m!(n-k-m)!}mkf(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m} $$ The first and fourth terms cancel. To see why, change the dummy index on the fourth term from $m$ to $m+1$ (similar to the above arguments). Let's write the second and third summations without the second summation notation (I kept the notation since there's so much index maneuvering in the problem). $$\sum\limits_{k=i+1}^{j-1}\sum\limits_{m=j-k-1}^{j-k-1} \frac{n!}{k!m!(n-k-m)!}k(n-k-m)f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{m}[1-F_{X}(v)]^{n-k-m-1}\\ -\sum\limits_{k=i}^{j-1}\sum\limits_{m=j-k-1}^{j-k-1} \frac{n!}{k!m!(n-k-m)!}m(n-k-m)f(v)f(u)[F_{X}(u)]^{k}[F_{X}(u)-F_{X}(u)]^{m-1}[1-F_{X}(v)]^{n-k-m-1} \\ = \sum\limits_{k=i+1}^{j-1} \frac{n!}{(k-1)!(j-k-1)!(n-j)!}f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{j-k-1}[1-F_{X}(v)]^{n-j} \\ -\sum\limits_{k=i}^{j-1} \frac{n!}{k!(j-k-2)!(n-j)!}f(v)f(u)[F_{X}(u)]^{k}[F_{X}(v)-F_{X}(u)]^{j-k-2}[1-F_{X}(v)]^{n-j} \\ =\sum\limits_{k=i+1}^{j-1} \frac{n!}{(k-1)!(j-k-1)!(n-j)!}f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{j-k-1}[1-F_{X}(v)]^{n-j} \\ -\sum\limits_{k=i}^{j-1} \frac{n!}{k!(j-k-1)!(n-j)!}(j-k-1)f(v)f(u)[F_{X}(u)]^{k}[F_{X}(v)-F_{X}(u)]^{j-k-2}[1-F_{X}(v)]^{n-j}$$ Hence, the very bottom term is zero when $k=j-1$, so we can rewrite this as $$\sum\limits_{k=i+1}^{j-1} \frac{n!}{(k-1)!(j-k-1)!(n-j)!}f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{j-k-1}[1-F_{X}(v)]^{n-j} \\ -\sum\limits_{k=i}^{j-2} \frac{n!}{k!(j-k-2)!(n-j)!}f(v)f(u)[F_{X}(u)]^{k}[F_{X}(v)-F_{X}(u)]^{j-k-2}[1-F_{X}(v)]^{n-j}$$ Change the dummy index on the bottom term from $k$ to $k-1$ and we have $$\sum\limits_{k=i+1}^{j-1} \frac{n!}{(k-1)!(j-k-1)!(n-j)!}f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{j-k-1}[1-F_{X}(v)]^{n-j} \\ -\sum\limits_{k=i+1}^{j-1} \frac{n!}{(k-1)!(j-k-1)!(n-j)!}f(v)f(u)[F_{X}(u)]^{k-1}[F_{X}(v)-F_{X}(u)]^{j-k-1}[1-F_{X}(v)]^{n-j} \\ = 0$$ So all the terms cancel and we have our result. $$ f_{X_{(i)},X_{(j)}}(u,v)= \frac{n!}{(i-1)!(j-i-1)!(n-j)!}f(v)f(u)[F_{X}(u)]^{i-1}[F_{X}(v)-F_{X}(u)]^{j-i-1}[1-F_{X}(v)]^{n-j} $$