Is the moment generating function smooth by definition?

Question

So I'm going through Casella's Statistical Inference, and in Definition 2.3.6 he defines the moment generating function of a random variable $X$ with cdf $F_X$, denoted by $M_X(t)$, as $$M_X(t) = Ee^{tx},$$ provided that the expectation exists for $t$ in some neighborhood of $0$.

In Theorem 2.3.7, he then states that if $X$ has a mgf $M_X(t)$, then the nth moment of $X$ is equal to the nth derivative of $M_X(t)$ evaluated at $0$.

But why do all of these derivatives have to existed? Is it implicitly assumed without it being stated, or does it somehow just follow from the definition above? If so, how?

Answer 1

These derivatives exist as soon as the moment generating function is finite on some interval containing $0$ (i.e. $M_x(t)$ exists for all $t \in [0,\varepsilon)$ where $\varepsilon > 0$). Often such details are glossed over in textbooks because they can sometimes require some more advanced machinery to prove.

In this case, a straightforward proof is possible. For $t_0 \in [0, \varepsilon)$, we have $$\frac1t \mathbb{E}[e^{(t_0 + t)X} - e^{t_0X}] = \mathbb{E}[e^{t_0X}X] + o(t)\mathbb{E}[X^2 e^{t_oX}]$$ by Taylor's theorem. By Holder's inequality, $\mathbb{E}[X^2 e^{t_0 X}] \leq \mathbb{E}[X^{2q}]^{1/q} \mathbb{E}[ e^{p t_0 X} ]^{1/p}$ where $p > 1$ is chosen so that $t_0 p < \varepsilon$ and $1/q + 1/p = 1$.

Since the existence of the moment generating function on any interval containing $0$ implies the existence of moments of all orders, it follows that $\frac{d}{dt}M_X \big|_{t_0} = \mathbb{E}[e^{t_0X} X]$. Repeating this argument will give $$\frac{d^n}{dt^n}M_X \big|_{t_0} = \mathbb{E}[e^{t_0X} X^n]$$ since the presence of $X$ really doesn't make a difference. Evaluating at $t_0 = 0$ gives the result.

Answer 2

Suppose $M_X(t)$ is well defined in $[a,b]$. Let $t_0\in(a,b)$. We want to prove that $M_X(t)$ is smooth at $t_0$.

Let $\eta>0$ be such that $[t_0-2\eta,t_0+2\eta]\subset(a,b)$ and define $f(t,\omega):=e^{tX(\omega)}$. If for $k=1,2,....$ we can find $g_k\in L^1(d\omega)$ such that $|\frac{\partial^k f}{\partial t^k}(t,\omega)|\leq g_k(\omega)$ for all $t\in (t_0-\eta,t_0+\eta)$, then we are done since we can differentiate under the integral sign (essentially by the dominated convergence theorem).

To find $g_k$, simply observe that for $t\in(t_0-\eta,t_0+\eta)$ $$|\frac{\partial^k f}{\partial t^k}(t,\omega)|=|X|^ke^{tX} =e^{t_0 X}|X|^ke^{(t-t_0)X} \leq C(\eta) e^{t_0 X} e^{2\eta |X|}=:g_k,$$ since $|x|^ke^{(t-t_0)x} \leq C(\eta) e^{2\eta |x|}$ for some constant $C(\eta)$ for all $x\in \mathbb{R}$ (easy to check because the RHS goes to infinity faster and is always positive), and $g_k$ is integrable since $e^{t_0 X} e^{2\eta |X|}\leq e^{(t_0+2\eta)X}+e^{(t_0-2\eta)X}$, which is integrable by definition of $\eta$.

If you are only interested in proving smoothness in a neighbourhood of $0$, say in $(-\delta,\delta)$, the proof is simpler: from the fact that $e^{\delta|X|}\leq e^{\delta X}+e^{-\delta X}$ is integrable, by the dominated convergence theorem you can immediately conclude that $\mathbb{E}(e^{tX})$ can be expressed as a power series in $t$ (Taylor-expanding the exponential). Then smoothness follows from the property of power series.