How to formally derive this formula for Kolmogorov-Smirnov two-sample test D-statistic

29 Views Asked by At

I'm trying to wrap my head around Kolmogorov-Smirnov two-sample test. So, Kolmogorov-Smirnov test uses the following D-statistic: $$D_{n_1, n_2} = \sup\limits_{-\infty < x < +\infty} \left|\widehat{F}_{n_1}(x) - \widehat{F}_{n_2}(x) \right|,$$

where $\widehat{F}_{n_1}(x)$ and $\widehat{F}_{n_2}(x)$ are empirical distribution functions.

My questions is how we can go from the expression above to $$D_{n_1, n_2} = \max\limits_{1 \le k \le N} \left|\widehat{F}_{n_1}(z_{(k)}) - \widehat{F}_{n_2}(z_{(k)}) \right|,$$

where $z_{(k)}$ - k-th order statistic of a sample made by joining 2 initial samples, and $N$ is the sum of samples' sizes.

I think I understand why it's this way intuitively, but I doubt it's good enough for a formal proof. The way I see it, since both functions are bounded, we can substitute supremum for a maximum. And we can also only look through the elements of our samples (that's something that makes sense but I don't know why).

So if anyone can help with a more formal proof, I would appreciate it.