$X$ sub-Gaussian $\implies \text{Var}(X) \leq \sigma^2$

Question

$\newcommand{\V}{\text{Var}}$ $\newcommand{\E}{\mathbb E}$

Definition:

A mean zero random variable $X$ is $\sigma$ sub-Gaussian if for all $\lambda \in \mathbb R$,

\begin{align} \E\left[\exp\left(\lambda X\right)\right] \leq \exp\left(\frac{\lambda^2 \sigma^2}{2}\right). \end{align}

Problem:

If $X$ has mean zero and is $\sigma$ sub-Gaussian, show that $\V(X) \leq \sigma^2$.

My attempt:

I am looking at Proposition 2.1 in these notes and 3.2 in these notes, which both supposedly solve this problem, but my analysis chops aren't quite good enough to justify what I'm doing.

We have that for all $\lambda \in \mathbb R$,

\begin{align} \E\left[\exp\left(\lambda X\right)\right] &\leq \exp\left( \frac{\lambda^2 \sigma^2}{2}\right)\\ \implies 1 + \lambda \E[X] + \frac{\lambda^2 \E[X^2]}{2!} &\leq 1 + \frac{\lambda^2 \sigma^2}{2} + o(\lambda^2) \end{align}

as $\lambda \to 0$. After subtracting and canceling certain terms from both sides of the inequality, letting $\lambda \to 0$, and using that $\E[X] = 0$ by assumption, we get that

\begin{align} \E[X^2] &\leq \sigma^2\\ \implies \V(X) &\leq \sigma^2 \end{align}

since in this case $\E[X^2] = \V(X)$.

Question:

Is this a legitimate solution? I am not totally comfortable with the manipulations or little-$o$ notation. I did a bunch of Googling but didn't find anything that seemed to help.

Thanks.

Answer 1

$\newcommand{\V}{\text{Var}}$ $\newcommand{\E}{\mathbb E}$

This is a simpler attempt, and I think it works. It's mostly just series/power series manipulations.

We have that for all $\lambda \in \mathbb R$,

\begin{align} \E\left[\exp\left(\lambda X\right)\right] &\leq \exp\left( \frac{\lambda^2 \sigma^2}{2}\right)\\ \implies 1 + \lambda \E[X] + \frac{\lambda^2 \E[X^2]}{2!} + \dots &\leq 1 + \frac{\lambda^2 \sigma^2}{2} + \left. \left(\frac{\lambda^2 \sigma^2}{2}\right)^2 \right/ 2! + \dots\\ \implies \frac{\lambda^2 \E[X^2]}{2!} + \dots &\leq \frac{\lambda^2 \sigma^2}{2} + \left. \left(\frac{\lambda^2 \sigma^2}{2}\right)^2 \right/ 2! + \dots\\ \implies \E[X^2] + \dots &\leq \sigma^2 + \frac{2}{\lambda^2} \cdot \left. \left(\frac{\lambda^2 \sigma^2}{2}\right)^2 \right/ 2! + \dots\\ \implies \lim_{\lambda \to 0} \left[ \E[X^2] + \dots \right] &\leq \lim_{\lambda \to 0} \left[ \sigma^2 + \frac{2}{\lambda^2} \cdot \left. \left(\frac{\lambda^2 \sigma^2}{2}\right)^2 \right/ 2! + \dots \right]\\ \implies \E[X^2] &\leq \sigma^2\\ \implies \V(X) &\leq \sigma^2. \end{align}

In the fourth line I divide by $2/\lambda^2$, so at that point, the inequality only holds for $\lambda \in \mathbb R \setminus \{ 0 \}$. However, the functions on both sides are power series, thus continuous, so the limits exist everywhere and there is no problem.

Answer 2

The second inequality in your attempt says $$E(X^2) \leq \sigma^2+o(1),\quad \lambda \to 0$$ In order to conclude $E(X^2)\leq \sigma^2$, you need to be more explicit with the Landau notation, namely you need to show that the function which is $o(1)$ is non-negative for all $\lambda$ sufficiently close to $0$. At least for me, I do not see any (obvious) reason why this needs to be true.

Edit: See comments

A correct proof can be given by using the identity $$f''(x) = \lim_{h\to 0}\frac{f(x+h)+f(x-h)-2f(x)}{h^2},\quad f\in C^2$$

Answer 3

That is valid, but you may want to expand on the

After subtracting and canceling certain terms from both sides of the inequality

since that's the crux of the argument.

Answer 4

Clarify some rationale behind the steps and it should be good.

At the step that you arrived

$$ \lambda^2 \mathbb{E}[X^2] \leq \lambda^2 \sigma^2 + \lambda^2 R(\lambda) $$

The inequality still holds for every $\lambda \in \mathbb{R}$.

However when you divide both sides by $\lambda^2$, we require $\lambda \neq 0$. Therefore at the step $$ \mathbb{E}[X^2] \leq \sigma^2 + R(\lambda) $$ it only holds for $\lambda \in \mathbb{R}\backslash\{0\}$

Despite we cannot directly put $\lambda = 0$ into above inequality, we can still pass the limit $\lambda \to 0$ and the inequality still holds - provided that such limit exist. $R$ is nice enough (continuous at $0$) in this case so no need to worry about it.

The only thing need be note is that when we pass the limit, if the original inequality is strict, we will need to add the equality condition and became not strict anymore. But since the original inequality is not strict, there is no further change in the result:

$$ \mathbb{E}[X^2] \leq \sigma^2 + \lim_{\lambda \to 0} R(\lambda) = \sigma^2$$

P.S.: In fact we can minimize $\lambda$ and obtain $$ \mathbb{E}[X^2] \leq \sigma^2 + \min_{\lambda \in \mathbb{R}\backslash\{0\}}R(\lambda) $$ This is stricter if $R$ is negative somewhere, but it is not required in this question.

Answer 5

If $k(\lambda)=\log E(e^{\lambda X})$ then $g(\lambda)=\lambda^2\frac{\sigma^2}{2}-k(\lambda)\geq 0$ for all $\lambda$, and $g(0)=0$ implies that $0$ is a minimum of $g$ and $g'(0)=0=k'(0)=E(X).$ If $g''(0)=0$ then $Var{X}=k''(0)=\sigma^2.$ If $g'(0)<0$ there exists $h>0$ such that $g'(\lambda)<0$ for $0<\lambda<h$ and $0$ would not be a minimum for $g$. Hence $g''(0)=\sigma^2-Var{X}\geq 0.$