Why uniform discrete pdf does not converge to continuous?

Question

Why a discrete probability distribution does not converge to a continuous when the number of support points grow to infinity?

Consider a uniform distribution with support in $[0, 1]$, if that is continuous, then pdf in each point in the support would be one, while if the distribution is discrete with (infinitely) many support points the pdf in each point would be zero. Why is that?

Is the concept of pdf inherently different in discrete and continuous cases? I am looking for an intuition about this fundamental question.

Answer 1

There are several ways to look at what's going on here - indeed, several ways to define what it would mean for such a convergence to occur. Let's look at a few:

1. The cdf of a continuous $U(0,\,1)$ distribution is $x$ on $[0,\,1]$, so it rises smoothly as a line. The cdf of its discrete counterpart looks more like a staircase. Does the staircase become the line? Here's a similar puzzle: the staircase consists of horizontal and vertical segments respectively summing to $1$ and $1$ so is of length $2$, so why is the straight path only of length $\sqrt{2}$?
2. The distribution which gives $0,\,\frac{1}{n},\,\cdots,\,1$ a probability of $\frac{1}{n+1}$ each has moment-generating fucntion $\frac{1}{n+1}\sum_{k=0}^ne^{tk/n}$. Is the $n\to\infty$ limit the expected $\frac{e^t-1}{t}$? Sure; the series tends to the integral $\int_0^1 e^{tx} dx=\frac{e^t-1}{t}$.
3. If you want to think of the probability mass functions of discrete distributions as if they were probability density functions, read about the Dirac delta. Our discrete distribution's "density" would be $\frac{1}{n+1}\sum_{k=0}^n\delta(x-k/n)$. Again, in the $n\to\infty$ limit the sum becomes an integral, $\int_0^1\delta(x-y)dy=1$ for $x\in [0,\,1]$.

Answer 2

Let, for every $n \geq 1$, $p_n$ is the uniform probability on $\left\{0,\frac{1}{n},\ldots,\frac{n-1}{n},1\right\}$.

Then $p_n$ converges in distribution to the uniform probability measure on $[0,1]$ ($\mu$), with the following (standard) meaning:

For every continuous function $f:[0,1] \rightarrow \mathbb{R}$, $\int{fdp_n} \rightarrow \int{fd\mu}$.