What is Moment Generating Function simply explained?

Question

I'm taking Econometrics this Trimester and there a quite few things I don't get yet, and one of them is MGF. I have tried Wikipedia etc. but seems to be written in a bit more advanced language. Any good explanation or link? Thanks

Answer 1

Suppose that $X$ is your random variable. Then the moment generating function is $M_X(t) = \mathbb{E}[e^{tX}]$, where $t$ is a real number, though not necessarily any real number (you might only be allowed to pick $t$ within a certain range).

There is no non-mathematical interpretation as to what a moment generating function is. It is purely a mathematical concept (it is related to Fourier transforms). That said, moment generating functions are useful since a random variable with some distribution has a unique moment generating function, and a moment generating function maps to some unique distribution. Thus, given a moment generating function, it is possible to identify the distribution it is associated with.

For example, suppose $X$ is a coin flip with probability $p$ of appearing heads, and equals 1 when heads, 0 when tails. The moment generating function can easily be calculated:

$$M_X(t) = \mathbb{E}[e^{tX}] = p e^{t \times 1} + (1 - p) e^{t \times 0} = p e^t + (1 - p)$$

I could even generalize this and say that when the coin lands heads-up, $X = A$ and when it lands tails-up, $X = B$ (instead of always 1 or 0) and the moment generating function would be:

$$M_X(t) = \mathbb{E}[e^{tX}] = p e^{At} + (1 - p) e^{Bt}$$

Suppose then I were told that the moment generating function for $Y$ is $M_Y(t) = \frac{1}{2} e^t + \frac{1}{2}$. A model of a fair coin that yields $Y = 1$ when the coin lands heads-up and $Y = 0$ when the coin lands tails-up would fit this moment generating function, so that must be the distribution of $Y$.

Moment generating functions are used extensively since they allow for easily computing the distribution of transformations of random variables. Suppose I wanted to identify the distribution of the sum of two independent random variables $X$ and $Y$; this could be a painful computation, but thanks to moment generating functions, I can say:

$$M_{X + Y}(t) = \mathbb{E}[e^{t(X + Y)}] = \mathbb{E}[e^{tX}e^{tY}] = \mathbb{E}[e^{tX}] \mathbb{E}[e^{tY}] = M_X(t) M_Y(t)$$

That is, just multiply the random variables moment generating functions together and you get the moment generating function for the distribution of their sum, which you can use to identify their distribution. (Exercise: Look up the moment generating function of Normally-distributed random variables with mean $\mu$ and variance $\sigma^2$, and use this technique to find the distribution of $X + Y$ when $X \sim N(1, 1)$ and $Y \sim N(2, 3)$, with $X$ and $Y$ independent.)

Moment generating functions get their name from the fact that they can be used to compute the expected values of moments of random variables (where the $n$th moment of a random variable $X$ is $\mathbb{E}[X^n]$). For example, to get the 1st moment, first take the derivative of the moment generating function:

$$M'_X(t) = \frac{d}{dt}\mathbb{E}[e^{tX}] = \mathbb{E}[\frac{d}{dt}e^{tX}] = \mathbb{E}[Xe^{tX}]$$

Set $t = 0$ to get $M'_X(0)=\mathbb{E}[X]$. Take more derivatives to get more moments. (Exercise: Use the moment generating function of a standard Normal random variable to compute $\mathbb{E}[X]$, $\mathbb{E}[X^2]$, $\mathbb{E}[X^3]$, and $\mathbb{E}[X^4]$.)

Moment generating functions have a lot of useful properties but I'll let you look those up. Often moment generating functions for common distributions are distributed in tables, already computed for you. The Wikipedia article has one.