Problem: Taken from A Probabilistic Theory of Pattern Recognition by Devroye et al.
Let $U_{(1)},\dots,U_{(n)}$ be order statistics of $n$ i.i.d. $\text{Unif}(0,1)$ variables. Show that $$\mathbb{E}[U_{(k)}^t]=\frac{\Gamma(k+t)\Gamma(n+1)}{\Gamma(k)\Gamma(n+1+t)}$$ for $t>0$ and $k\in[n]$.
My Attempt: For the $k$th order statistic, we find the distribution function first. Let $U_1,\dots,U_n$ denote the unordered Uniform random variables and furthermore let $\mathbb{I}\{U_i\leq u\}$ denote the indicator variable that $U_i\leq u$. Then we have the distribution is given by
$$\mathbb{P}(U_{(k)}\leq u)=\mathbb{P}\left(\bigcup\limits_{i=k}^n \left\{\sum\limits_{j=1}^n \mathbb{I}\{U_j\leq u\}=i\right\}\right)$$
Notice that the events under the union are trivially disjoint - for instance, $k$ occurrences of the uniform variables being less than $u$ implies that $k+1$ occurrences is impossible (I don't know why I explicated this). Thus by additivity of $\mathbb{P}$,
$$\mathbb{P}(U_{(k)}\leq u)=\sum\limits_{i=k}^n \mathbb{P}\left(\sum\limits_{j=1}^n \mathbb{I}\{U_j\leq u\}=i\right).$$
Notice the sum of indicators is binomial and each indicator has probability $u$ of success, so
$$\mathbb{P}(U_{(k)}\leq u)=\sum\limits_{i=k}^n \binom{n}{i} u^i(1-u)^{n-i}.$$
As the order statistics are positive, we can use the formula $\mathbb{E}[X^k]=\int kx^{k-1}\overline{F}_X(x)\;dx$. Thus we have for $t>0$,
$$\mathbb{E}[U_{(k)}^t]=\int\limits_{0}^1 tu^{t-1}\left(1-\sum\limits_{i=k}^n \binom{n}{i} u^i(1-u)^{n-i}\right)\;du.$$
Distributing $tu^{t-1}$ and recognizing the second term as the beta function yields that
$$\mathbb{E}[U_{(k)}^t]=1-\sum\limits_{i=k}^n \binom{n}{i}t\int\limits_0^1 u^{t+i-1}(1-u)^{n-i}\;du$$
$$=1-\sum\limits_{i=k}^n \binom{n}{i}t\frac{\Gamma(t+i)\Gamma(n-i+1)}{\Gamma(n+t+1)}.$$
We can simplify furthermore to see that
$$\mathbb{E}[U_{(k)}^t]=1-\frac{t\Gamma(n+1)}{\Gamma(n+t+1)}\sum\limits_{i=k}^n \frac{\Gamma(t+i)}{\Gamma(i+1)}.$$
Thus we need to show
$$1-\frac{\Gamma(k+t)\Gamma(n+1)}{\Gamma(k)\Gamma(n+1+t)}=\frac{t\Gamma(n+1)}{\Gamma(n+t+1)}\sum\limits_{i=k}^n \frac{\Gamma(t+i)}{\Gamma(i+1)}.$$
Question: Is this statement even true (and if so, how could I continue) or did I lose myself in the algebra? Hints and tangential solutions are also appreciated!
Yes, the statement is true.
$$\begin{align} \sum_{i=k}^n \frac{\Gamma(t+i)}{\Gamma(i+1)} &= (t-1)!\sum_{i=k}^n \frac{(i+t-1)!}{i!(t-1)!} \\ &= (t-1)! \sum_{i=k}^n \binom{i+t-1}{t-1} \\ &= (t-1)! \left( \sum_{i=0}^n \binom{i+t-1}{t-1} - \sum_{i=0}^{k-1} \binom{i+t-1}{t-1} \right) \\ &= (t-1)! \left( \binom{n+t}{t} - \binom{k+t-1}{t} \right), \tag{1} \end{align}$$
where we have employed the hockey-stick identity $$\sum_{j=0}^{n-r} \binom{j+r}{r} = \binom{n+1}{r+1}.$$
Consequently $$\begin{align} \frac{t \Gamma(n+1)}{\Gamma(n+t+1)} \sum_{i=k}^n \frac{\Gamma(t+i)}{\Gamma(i+1)} &= \frac{1}{\binom{n+t}{t}} \left( \binom{n+t}{t} - \binom{k+t-1}{t}\right) \\ &= 1 - \frac{(k+t-1)!t! n!}{t!(k-1)!(n+t)!} \\ &= 1 - \frac{\Gamma(k+t) \Gamma(n+1)}{\Gamma(k) \Gamma(n+t+1)} \tag{2} \end{align}$$
as claimed.
However, this is an incredibly tedious and inefficient way to prove the desired result. It is much, much easier to work with the probability density of the order statistic. As noted in the Wikipedia article, the density of the $k^{\rm th}$ order statistic is
$$f_{X_{(k)}}(x) = \frac{n!}{(k-1)! (n-k)!} f_X(x) F_X(x)^{k-1} (1 - F_X(x))^{n-k}. \tag{3}$$ In the case where $X = U$ is uniform on $[0,1]$, this implies $f_U(u) = 1$ and $F_U(u) = u$, hence $(3)$ simplifies to
$$f_{U_{(k)}}(u) = \frac{n!}{(k-1)! (n-k)!} u^{k-1} (1-u)^{n-k} = \frac{\Gamma(n+1)}{\Gamma(k)\Gamma(n-k+1)} u^{k-1} (1-u)^{(n-k+1)-1} \tag{4}$$
which demonstrates that $U_{(k)} \sim \operatorname{Beta}(k,n-k+1)$. Thus the $t^{\rm th}$ moment is
$$\begin{align} \operatorname{E}[U_{(k)}^t] &= \int_{u=0}^1 u^t f_{U_{(k)}}(u) \, du \\ &= \frac{\Gamma(n+1)}{\Gamma(k) \Gamma(n-k+1)}\int_{u=0}^1 u^{t+k-1} (1-u)^{n-k} \, du \\ &= \frac{\Gamma(n+1)}{\Gamma(k)} \cdot \frac{\Gamma(t+k)}{\Gamma(t+n+1)} \tag{5} \end{align}$$
and we are done.