Smoothness of the logarithm of moment generating functions

154 Views Asked by At

This is Problem $2.6$ from the text on concentration inequalities by Boucheron, Lugosi and Massart.

Let $Z$ be a real-valued random variable. Show that the set $$S = \{\lambda \geqslant 0 \mid \mathbb{E}[e^{\lambda Z}] < \infty\}$$ is an interval with left end point equal to $0$. Let $b = \sup S$. Show that $$\psi_Z(\lambda) = \log \mathbb{E}[e^{\lambda Z}]$$ is convex and infinitely many times differentiable on $I = (0,b)$.

I was able to show that $S$ is an interval and that $\psi_Z$ is convex on $[0, \sup S)$ using the convexity of the maps $f_a :x \mapsto e^{ax}$ for $a\in \mathbb{R}$. The dominated convergence theorem can be used to establish the smoothness of $M_Z :\lambda \mapsto \mathbb{E}[e^{\lambda X}]$ (and hence $\psi_Z$) in a neighbourhood $N := (-\delta, \delta)$ of the origin provided $M_Z$ if finite on $N$, by expanding $M_Z$ in a power series centered around the origin. This argument can be made to work in our setting provided we show that $M_Z$ is finite on $-S := \{-s \mid s \in S\}$. However, this is not always the case -- suppose that $X \sim \text{Exp}(a)$ and consider $Z = -X$. Then $M_Z$ is finite on $(-a, +\infty)$.

How should one go about proving smoothness? Any help is appreciated.

1

There are 1 best solutions below

0
On BEST ANSWER

$\def\e{\mathrm{e}}\def\paren#1{\left(#1\right)}$First, note that $E(\e^{λZ}) > 0$ for any $λ \in S$ (otherwise $Z = -∞$ almost surely), so proving the smoothness of $ψ_Z(λ)$ is equivalent to proving that of $M_Z(λ)$. In order to utilize the dominated convergence theorem, the following lemma is needed to prove that $E(|Z|^n \e^{λZ}) < +∞$.

Lemma: For any $x, μ, ν > 0$,$$ x^ν \leqslant \paren{ \frac{ν}{\e μ} }^ν \e^{μx}. $$

Proof: Define $f(x) = x^ν \e^{-μx}$ for $x > 0$, then\begin{gather*} (\ln f(x))' = \frac{ν}{x} - μ \Longrightarrow \sup_{x > 0} f(x) = f\paren{ \frac{ν}{μ} } = \paren{ \frac{ν}{\e μ} }^ν. \tag*{$\Box$} \end{gather*}

Now return to the question. For any $λ \in (0, b)$ and $n \in \mathbb{N}$, since $\dfrac{\partial^n}{\partial λ^n}(\e^{λx}) = λ^n \e^{λx}$ and\begin{align*} &\mathrel{\phantom=} E(|Z|^n \e^{λZ})\\ &= E(|Z|^n \e^{-λ|Z|} I_{\{Z < 0\}}) + E(|Z|^n \e^{λ|Z|} I_{\{Z > 0\}})\\ &\leqslant E\paren{ \paren{ \frac{n}{\e λ} }^n \e^{λ|Z|} · \e^{-λ|Z|} I_{\{Z < 0\}} } + E\paren{ \paren{ \frac{n}{\e · \frac{1}{2} (b - λ)} }^n \e^{\frac{1}{2} (b - λ)|Z|} · \e^{λ|Z|} I_{\{Z > 0\}} }\\ &\leqslant \paren{ \frac{n}{\e λ} }^n + \paren{ \frac{n}{\e · \frac{1}{2} (b - λ)} }^n E(\e^{\frac{1}{2} (b + λ)|Z|}) < +∞, \end{align*} so the dominated convergence theorem shows that$$ \frac{\partial^n}{\partial λ^n}(E(\e^{λZ})) = E\paren{ \frac{\partial^n}{\partial λ^n}(\e^{λZ}) } = E(Z^n \e^{λZ}). $$ Therefore, $M_Z(λ)$ (and thus $ψ_Z(λ)$) is smooth.