Finite discrete approximation to the normal distribution

Question

I wish to derive a finite (that is, which has a finite support) discrete approximation to a normal distribution, with the following considerations:

1. It should have exactly the same mean and variance
2. It must be symmetric
3. It should resemble the normal distribution in some sense (unimodal pmf, etc.)
4. It should be discrete and finite (having finite support), with a pre-determined set which contains the support (for example, the integers).

Naive attempt

Here's a naive attempt. Suppose we wish to give an approximation to $\mathcal{N}(\mu,\sigma^2)$. Let the support be $S=\left[\lfloor \mu-d\sigma \rfloor,\lceil \mu+d\sigma\rceil\right]$ (for some natural $d$, perhaps $3$), and define the following pmf: \begin{equation*} f(k) = \begin{cases} \Phi_{\mu,\sigma^2}\left(k+\frac{1}{2}\right) & k = \min{S}\\ \Phi_{\mu,\sigma^2}\left(k+\frac{1}{2}\right) - \Phi_{\mu,\sigma^2}\left(k-\frac{1}{2}\right) & \min{S} < k < \max{S}\\ 1 - \Phi_{\mu,\sigma^2}\left(k-\frac{1}{2}\right) & k = \max{S} \end{cases} \end{equation*} where $\Phi_{\mu,\sigma^2}$ is the cdf of $\mathcal{N}(\mu,\sigma^2)$. This is a legitimate pmf (sums to $1$), it is symmetric, unimodal, discrete and finite, and has mean $\mu$ -- but it does not have variance $\sigma^2$ (I think it always has a larger variance).

Can you fix this naive solution somehow?

Answer 1

Just normalize the expected hitting values (# of times process obtains a given integer) of a simple one-dimensional random walk on $\mathbb{Z}$.

This is equivalent to shifting the binomial distribution $B(n,\frac{1}{2})-\frac{n}{2}$, modulo any little odd/even parity issues. Which is why the normal distribution, $N(n,\frac{n}{4})$, is used as an approximation for the binomial distribution.

It is easy to get intuition for why this works by glancing at Pascal's Triangle (the binomial expansion). In other words, if you have say 7 ordered objects which you want to approximate a normal distribution over selecting, just normalize the coefficients of the binomial expansion with exponent $n=7$. Or normalize the seventh row of Pascal's Triangle.

Answer 2

Let me first rephrase your question to make it more concrete to motivate the solution.

The normal distribution is a ubiquitous probability distribution because given a mean and a variance, a normal distribution yields the maximum entropy among all continuous pdfs (see https://en.wikipedia.org/wiki/Maximum_entropy_probability_distribution).

Now we seek to find the maximum entropy discrete pdf given a mean, variance, and a range (min, max, step size).

I claim the general solution is a beta-binomial distribution (https://en.wikipedia.org/wiki/Beta-binomial_distribution).

[Proof left as exercise for reader]

Note that ${\displaystyle \mathrm {BetaBin} (n,\alpha ,\beta )}$ has support $x ∈ { 0, …, n }$, mean ${\frac {n\alpha }{\alpha +\beta }}$, and variance ${\frac {n\alpha \beta (\alpha +\beta +n)}{(\alpha +\beta )^{2}(\alpha +\beta +1)}}$

Let's work out an example problem for clarity. Suppose you took a test and your professor says the mean was 75, standard deviation was 15, with a min of 0 and max of 100.

The solution must be of the form ${\displaystyle \mathrm {BetaBin} (100,\alpha ,\beta )}$ to have a support of the integers from 0 to 100. Then $\alpha = 3\beta$ for the mean to be 75. Now $225 = {\frac {300\beta^2 (4\beta +100)}{(4\beta )^{2}(4\beta +1)}}$ yields $\beta = 2$.

Therefore the maximum entropy distribution of test scores is ${\displaystyle \mathrm {BetaBin} (100,6,2)}$plot of pdf

Answer 3

Restating the question, what probability mass function with finite support $ P(X=k) $ most excellently approximates the normal distribution $ X \sim \mathcal{N}(\mu, {\sigma}^2) $; mimicking its symmetrical shape, statistical measures, and general behavior?

The Normal Mass Function

$$ P(X=k) = \frac{g(k)}{C} $$

$$ g(k) = e^{\frac{-(k-\mu)^2}{2{\sigma}^2}} $$

$$ C = \sum_{i=1}^{n} g(k_i) \approx \sqrt{2 \pi {\sigma}^2 } $$

The Normal Mass Function (NMF) is a PMF that is exactly proportional to the Normal PDF, and if given an adequate support, is virtually the same function. It performs so well, and shares such a tight connection with its continuous brother, that I had to name it authoritatively. But some limitations must be respected in order to achieve admirable performance, as follows.

The support of the Normal Mass Function must consist of equally spaced points.

$$ \delta = k_{i+1} - k_i $$

I have three free parameters in creating the support $ \{k_1, \ldots, k_n\} $. The parameters are the spacing, the start, and the number of points, namely $ \delta $, $ k_1 $ and $ n $. These parameters may be chosen for one to one comparison with other discrete distributions. However, for best agreement to the normal distribution, the support should not only have equally spaced points, but also be defined symmetrically, with $ \text{median}(k_i) = \mu $, or equivalently:

$$ k_1 = \mu - (n-1) \delta / 2 $$ $$ k_n = \mu + (n-1) \delta / 2 $$

This refined support in context of the same function defines the Symmetric Normal Mass Function.

I have two free parameters in creating the symmetric support: the spacing, and the number of points, namely $ \delta $ and $ n $. If $ (n-1) \delta / \sigma \ge 12 $ and $ \sigma / \delta \ge 1 $ then I have an acceptable support: the Symmetric Normal Mass Function agrees closely with the Normal Distribution. An acceptable support spans at least 6 standard deviations away from the mean (in both directions) and has at least one point per standard deviation.

By these rules, the minimum acceptable support has 13 points spaced at 1 point per standard deviation. Given the minimum acceptable support, the Symmetric Normal Mass Function compared to the Normal Distribution, has

• identical mean,
• approximate variance (low, within 2.2 parts per 10 million),
• identical skewness, and
• approximate kurtosis (high, within 2.4 parts per million).

$$ \mu, \sigma^2, \frac{\mu_3}{\sigma^3}, \frac{\mu_4}{\sigma^4} \approx \mu, \sigma^2, 0, 3 $$

Except for this minor decimal disagreement, the Symmetric Normal Mass Function meets all 4 requirements of the original question marvelously. The approximation improves as the support's span and density increases relative to the standard deviation.

I expound further about why the Normal Mass Function is the best choice as the discrete variant of the Normal Distribution in a linked article.