The intuition behind gamma distribution

Question

What is the intuition behind gamma distribution?

For instance, I understand how to "construct" Gaussian distribution. This is my intuition:

1. Bernoulli distribution - which is simple concept
2. A sequence of Bernoulli trials is a Binomial distribution. I understand how binomial coefficient is constructed
3. Using Stirling approximation we can deduce Gaussian distribution

Hence I understand that shape of a Gaussian distribution is determined by the binomial coefficients and so on.

How can Gamma distribution be derived step by step using relatively simple concepts?

Answer 1

The short answer is "prove the pdf is non-negative & integrates to 1", since that guarantees we're dealing with a probability distribution. But if you're looking for a motivation for why anything would be Gamma-distributed, here's the idea.

The $\alpha =1$ special case, with pdf $e^{-\beta x}$ for $x>0$, is the so-called exponential distribution, which concerns the lifetime of something that has the same risk of failure in a given time period regardless of its history (we call this memorylessness), e.g. a radioactive atom. (The mean lifetime is $\beta^{-1}$.) If $\alpha$ is a positive integer, you have the distribution obtained by summing $\alpha$ exponential iids. In general $\alpha$ can be any positive real, but that's just a continuous generalisation.

Answer 2

I understand the exponential distribution in a following way:

$$e^{-\lambda \cdot x} = \lim_{N \to \infty} (1 - \frac{\lambda}{N})^{N \cdot x}$$

Where:

$\lambda$ - number of events per one time unit (denoted by $1$)

$N$ - tends to infinity. It splits whole time unit into $N$ small intervals each of a length of $\frac{1}{N}$, such that only one event can occur within this small interval

$\frac{\lambda}{N}$ - is a probability of an event within one small time frame. Each time frame is a Bernoulli trial: event - success, no event - failure.

$(1 - \frac{\lambda}{N})$ - probability of "failure" - no event

$N \cdot x$ - is a number of consecutive "failures". Where $x$ is a part of 1 interval, let $N = 1000$ - some big number, then half of the interval ($x = 0.5$) would be $N \cdot x = 1000 * 0.5 = 500$ small time frames

$(1 - \frac{\lambda}{N})^{N \cdot x}$ - probability of $N \cdot x$ consecutive failures, or probability that the event will not occur $x$ amount of time

Answer 3

The following is some intuition behind the formula.

The Gamma distribution answer the question: 'How long do we have to wait between $a$ events with exponential wait times of $\frac{1}{B}e^{-t/B}$?'

For $a=1$ it's distributed as $\frac{1}{B}e^{-t/B}$, we already know that, it's just the exponential distribution.

How long will we have to wait for two events? In other words, for $a=2$?

Let's talk about the likleyhood of waiting $5$ seconds ($T=5$) if we have to wait for two exponential events ($a=2$)

Roughly $P(T=5| a=2) = \sum_0^5 P(T = i| a=1)P(T = 5-i|a=1)$

We can slice time into smaller slices until the above is an integral.

$p(t=5) =\int_0^5 p(T = i | a=1)p(T = 5-i| a=1)di = \frac{1}{B^2}\int_0^5 e^{-i/B}e^{(-5-i)/B} di=\frac{1}{B^2}\int_0^5 e^{-5/B} di= \frac{1}{B^2}e^{-5/B} \int_0^5 di = 5\frac{1}{B^2}e^{-5/B} $

replacing $5$ with '$x$' we get:

$p(t=x|a=2) = x\frac{1}{B^2}e^{-x/B}$

Let's now increase to $a = 3$

In the following the double integral is over the area $i+j < 5$

$p(t=5) = \int \int p(T = i| a=1 )p(T = j| a=1)p(T=5-i-j| a=1)didj = \frac{1}{B^3}\int \int e^{-i/B}e^{-j/B}e^{-(5-i-j)/B} didj=\frac{1}{B^3} e^{-5/B} \int\int didj= \frac{1}{B^3}e^{-5/B} \int\int didj = 12.5\frac{1}{B^3}e^{-5/B} $

Where $12.5=\frac{5^2}{2}$ is the area of the integration triangle with side lengths 5.

replacing $5$ with $x$ we get:

$p(t=x|a=3) = \frac{x^2}{2}\frac{1}{B^3}e^{-x/B}$

As we increase to higher $a$'s we will be integrating over simplexs of dimension $a-1$.

Therefore:

$p(t=x|a=n) = \frac{x^{n-1}}{(n-1)!}\frac{1}{B^n}e^{-x/B} = \frac{x^{n-1}}{\Gamma(n-1)B^n}e^{-x/B}$