I was trying to show that the sum of i.i.d. normals is normal. Since I had no idea how to do it I started looking online and found out about moment generating functions (MGF). Most proofs about that sum concluded the argument saying that because the MGF of the sum equals the MGF of a normal, then the sum must be a normal (example: look at last phrase of the document).
Question: Why is it true? If two random variables have the same MGF, then they must have the same distribution? Is there a one-to-one correspondence between distributions and MGFs? Can someone shed some light on this?
Thanks for helping!
It is possible and instructive to prove the theorem directly, because the path through characteristic functions (or moment generating functions) is actually a very long one that requires far more than one might think, including complex analysis and dominated convergence theorem from measure theory. $\def\pp{\mathbb{P}}$
Notation
Given random variable $X$, let "$f_X$" denote the density function for $X$.
Convolution
Given continuous real-valued independent random variables $X,Y$, and $Z = X+Y$:
For any $v \in \mathbb{R}$:
$\pp( Z \le v ) = \pp( Y \le v-X ) = \int_{-\infty}^\infty \int_{-\infty}^{v-x} f_X(x) f_Y(y)\ dy\ dx$
$= \int_{-\infty}^\infty \int_{-\infty}^{v} f_X(x) f_Y(y-x)\ dy\ dx$
$= \int_{-\infty}^{v} \int_{-\infty}^\infty f_X(x) f_Y(y-x)\ dx\ dy$ [The swap is valid since the integrand is non-negative].
Thus $f_{Z}(v) = \frac{d}{dv} \pp( Z \le v ) = \int_{-\infty}^\infty f_X(x) f_Y(v-x)\ dx$ [by FTC].
Sum of normal random variables
Given independent random variables $X \sim N(a,s^2)$ and $Y \sim N(b,t^2)$, and $Z = X+Y$:
Let $W = Z - (a+b) = (X-a) + (Y-b)$.
For any $v \in \mathbb{R}$:
$f_W(v) = \int_{-\infty}^\infty \frac{1}{\sqrt{2πs^2}} \exp(-\frac{x^2}{2s^2}) \frac{1}{\sqrt{2πt^2}} \exp(-\frac{(v-x)^2}{2t^2})\ dx$
$= \int_{-\infty}^\infty \frac{1}{\sqrt{2πs^2}} \frac{1}{\sqrt{2πt^2}} \exp(-\frac{ x^2 t^2 + (v-x)^2 s^2 }{ 2 s^2 t^2 })\ dx$
$= \int_{-\infty}^\infty \frac{1}{\sqrt{2πs^2}} \frac{1}{\sqrt{2πt^2}} \exp(-\frac{ (s^2+t^2) x^2 - 2vx s^2 + v^2 s^2 }{ 2 s^2 t^2 })\ dx$
$= \int_{-\infty}^\infty \frac{1}{\sqrt{2πs^2}} \frac{1}{\sqrt{2πt^2}} \exp(-\frac{ ( (s^2+t^2) x - v s^2 )^2 + v^2 s^2 t^2 }{ 2 s^2 t^2 (s^2+t^2) })\ dx$. [by completing the square]
$= \frac{1}{\sqrt{2π(s^2+t^2)}} \exp(-\frac{ v^2 }{ 2 (s^2+t^2) } ) \int_{-\infty}^\infty \frac{1}{\sqrt{2πc}} \exp(-\frac{ ( x - v \frac{s^2}{s^2+t^2} )^2 }{ 2 c })\ dx$ where $c = \frac{s^2t^2}{s^2+t^2}$
$= \frac{1}{\sqrt{2π(s^2+t^2)}} \exp(-\frac{ v^2 }{ 2 (s^2+t^2) } )$ [the above known integral motivates the previous steps].
Therefore $W \sim N(0,s^2+t^2)$ and hence $Z \sim N(a+b,s^2+t^2)$.