Suppose $X_1,..., X_n$ are i.i.d. continuous r.v. with distribution function $F(x)$, and density function $f(x)$.
$X_{(1)}<\cdots<X_{(n)}$ are the order statistics.
I've already showed that the distribution function of $X_{(k)}$ is
$G_k(x)=\sum^n_{j=k} {n \choose j}F(x)^j(1-F(x))^{n-j}$.
Now I'm asked to prove that the density function of $X_{(k)}$ is
$g_k(x)= \frac{n!}{(n-k)!(k-1)!}F(x)^{k-1}(1-F(x))^{n-k}f(x)$.
I'm supposed to do the derivative of $G_k$ w.r.t $x$ and obtain the density above.
However, I get to $g_k(x)=\sum^n_{j=k} {n \choose j}\cdot \left(jF(x)^{j-1}(1-F(x))^{n-j}f(x)-F(x)^{j}(n-j)(1-F(x))^{n-j-1}f(x) \right)=\sum^n_{j=k} {n \choose j} F(x)^{j-1}(1-F(x))^{n-j-1}f(x)(j-nF(x))$
where I remain stuck...
Any help would be appreciated.
The distribution function is correct:
$$G_k(x)=\sum^n_{j=k} {n \choose j}F(x)^j(1-F(x))^{n-j}$$
The correct density function is (no sum)
$$g_k(x)=\frac{n!}{(n-k)!(k-1)!}F(x)^{k-1}(1-F(x))^{n-k}f(x).$$
Finding the density by directly differentiating the sum is difficult. Alternatively, the distribution function can be shown to have an integral representation in terms of the beta function
$$G_k(x) = \frac{n!}{(n-k)!(k-1)!}\int_{0}^{F(x)}t^{k-1}(1-t)^{n-k}\,dt,$$
and the density function is obtained directly by taking the derivative of the integral using Leibniz's rule.
We can derive the integral representation by induction.
Let
$$H(k,p) = \sum^n_{j=k} {n \choose j}p^j(1-p)^{n-j}$$
Then
$$H(0,p) = 1,\\H(1,p) = 1 - (1-p)^n= \frac{n!}{(n-1)!(1-1)!}\int_{0}^{p}t^{1-1}(1-t)^{n-1}\,dt$$
Assume
$$H(k-1,p) = \frac{n!}{(n-k+1)!(k-2)!}\int_{0}^{p}t^{k-2}(1-t)^{n-k+1}\,dt.$$
Integrate by parts to obtain
$$H(k-1,p) = {n \choose k-1}p^{k-1}(1-p)^{n-k+1}+\frac{n!}{(n-k)!(k-1)!}\int_{0}^{p}t^{k-1}(1-t)^{n-k}\,dt.$$
From the definition of $H$ as a sum we have,
$$H(k,p) = H(k-1,p) - {n \choose k-1}p^{k-1}(1-p)^{n-k+1}.$$
Hence,
$$H(k,p) = \frac{n!}{(n-k)!(k-1)!}\int_{0}^{p}t^{k-1}(1-t)^{n-k}\,dt.$$
Now substitute with $p = F(x)$ to obtain
$$G_k(x) =H(k,F(x))= \frac{n!}{(n-k)!(k-1)!}\int_{0}^{F(x)}t^{k-1}(1-t)^{n-k}\,dt.$$
Direct Derivation:
To get the density function directly, start with your formula
$$g_k(x)\\=\sum^n_{j=k} {n \choose j}\cdot \left(jF(x)^{j-1}(1-F(x))^{n-j}f(x)-F(x)^{j}(n-j)(1-F(x))^{n-j-1}f(x) \right)\\=f(x)\left[\sum^n_{j=k} j{n \choose j} F(x)^{j-1}(1-F(x))^{n-j}-\sum^{n-1}_{j=k} (n-j){n \choose j} F(x)^{j}(1-F(x))^{n-j-1}\right].$$
Note that
$$j{n \choose j}=n{n-1 \choose j-1},\\ (n-j){n \choose j}=n{n-1 \choose j}.$$
Then
$$g_k(x)\\=nf(x)\left[\sum^n_{j=k} {n-1 \choose j-1} F(x)^{j-1}(1-F(x))^{(n-1)-(j-1)}-\sum^{n-1}_{j=k} {n-1 \choose j} F(x)^{j}(1-F(x))^{(n-1)-j}\right].$$
The first sum is
$$\sum^n_{j=k} {n-1 \choose j-1} F(x)^{j-1}(1-F(x))^{(n-1)-(j-1)} \\={n-1 \choose k-1}F(x)^{k-1}(1-F(x))^{n-k} + \sum^n_{j=k+1} {n-1 \choose j-1} F(x)^{j-1}(1-F(x))^{(n-1)-(j-1)}$$
The second sum is
$$\sum^{n-1}_{j=k} {n-1 \choose j} F(x)^{j}(1-F(x))^{(n-1)-j)} \\=\sum^n_{j=k+1} {n-1 \choose j-1} F(x)^{j-1}(1-F(x))^{(n-1)-(j-1)}.$$
The remaining sums cancel leaving
$$g_k(x) = n{n-1 \choose k-1}F(x)^{k-1}(1-F(x))^{n-k}f(x)$$