How sub-Gaussian is a Truncated Normal?

587 Views Asked by At

We say the expectation random variable $X$ is $C$-subgussian to mean $$\mathbb E[e^{\lambda X - \mathbb E [X]}] \le e^{C\lambda^2}$$ for all $\lambda \in \mathbb R$. One slightly different definition is that

$$P(X-\mathbb E [X] > \lambda) \le e^{-D \lambda^2} \text{ and } P(X-\mathbb E [X] < \lambda) \le e^{-D \lambda^2}$$

for some $D$. The constants $C$ and $D$ can be bounded in terms of each other.

It is well-known a $N(0,\sigma^2)$ variable $X$ satisfies the above with $C= \sigma^2/2$ and $D = 1/2 \sigma^2$.

Suppose we truncate the variable to some finite interval $[a,b]$ that contains $E [X] $. For $Y$ the truncated variable it is straightforward to show say$$P(|Y-E [X] | < \lambda) \ge 1-2e^{-D \lambda^2}$$ for $D = 1/2 \sigma^2$ whenever $a \le -\lambda \le \lambda \le b $. This is because the PDF of $Y$ is just the PDF of $X$ set to zero outside $[a,b]$ and then scaled upwards to make it a probability distribution. However this does not give a subgaussian constant for $Y$ since $Y$ may not have expectation $\mathbb E [X] $.

Are there any ways to get a good subgaussian constant for $Y$ in terms of $a,b$? Note Hoeffding's lemma gives the constant $(b-a)^2/8$ but I would like something in terms of $\sigma^2$ that ideally tends to the original constant as $a,-b \to \infty$.

1

There are 1 best solutions below

0
On

I know it has been a while since you posted this but just in case you are still interested in the question, here is what I had found while working on a similar problem.

Let $X$ be a sub-Gaussian random variable with $\mathbb{E}[X] = \mu$ and $\mathbb{E}[\exp(\lambda (X - \mu))] \leq \exp(\lambda^2 \sigma^2/2)$. Define $Y = X \cdot \mathbb{I}\{[-(R+B), (R+B) \}$, where $\mathbb{I}\{A\}$ is the indicator function of the set $A$. In the above expression, $R >0$ and $B \geq |\mu|$ to ensure that $\mu$ lies in the interval.

We first obtain a bound on $|\mathbb{E}[X] - \mathbb{E}[Y]|$. We have, \begin{align*} |\mathbb{E}[X] - \mathbb{E}[Y]| & = |\mathbb{E}[X] - \mathbb{E}[X \cdot \mathbb{I}{[-(R+B), (R+B)]}]| \\ & = \left| \int_{-\infty}^{\infty} x f_{X}(x) \ \mathrm{d}x - \int_{-(R+B)}^{(R+B)} x f_{X}(x) \ \mathrm{d}x \right | \\ & \leq \left| \int_{-\infty}^{-(R+B)} x f_{X}(x) \ \mathrm{d}x \right| + \left| \int_{(R+B)}^{\infty} x f_{X}(x) \ \mathrm{d}x \right | \\ & \leq \left| \left[ x F_X(x) \right]_{-\infty}^{-(R+B)} - \int_{-\infty}^{-(R+B)} F_X(x) \ \mathrm{d}x \right| + \left| \left[ -x (1 - F_X(x)) \right]_{(R+B)}^{\infty} + \int_{(R+B)}^{\infty} (1 - F_{X}(x)) \ \mathrm{d}x \right | \\ & \leq \left| -(R+B) F_X(-(R+B)) - \int_{-\infty}^{-(R+B)} \Pr(X \leq x) \ \mathrm{d}x \right| + \\ & \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \left| (R+B) (1 - F_X((R+B))) + \int_{(R+B)}^{\infty} \Pr(X > x) \ \mathrm{d}x \right | \\ & \leq \left| (R+B)\Pr(X - \mu \leq -R - B - \mu) + \int_{-\infty}^{-(R+B+\mu)} \Pr(X - \mu \leq y) \ \mathrm{d}y \right| + \\ & \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \left| (R+B)\Pr(X - \mu \leq - R - B - \mu) + \int_{(R+B - \mu)}^{\infty} \Pr(X - \mu > z) \ \mathrm{d}z \right| \\ & \leq \left| (R+B) \Pr(X - \mu \leq -R) + \int_{-\infty}^{-R} \exp\left( - \frac{y^2}{2\sigma^2} \right) \ \mathrm{d}y \right| + \\ & \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \left| (R+B)\Pr(X - \mu \leq -R) + \int_{R}^{\infty} \exp\left( - \frac{z^2}{2\sigma^2} \right) \ \mathrm{d}z \right| \\ & \leq \left| (R+B) \exp\left( - \frac{R^2}{2\sigma^2} \right) + \sigma \Phi(-R/\sigma) \right| + \left| (R+B) \exp\left( - \frac{R^2}{2\sigma^2} \right) + \sigma(1 - \Phi(R/\sigma)) \right | \\ & \leq 2(R + B) \exp\left( - \frac{R^2}{2\sigma^2} \right) + 2 \sigma \Phi(-R/\sigma) \\ & \leq 2 \left( R + B + \frac{\sigma^2}{R \sqrt{2 \pi}} \right)\exp\left( - \frac{R^2}{2\sigma^2} \right) := t_0. \end{align*}

We now bound the moment generating function of $Y$. For any $\lambda \in \mathbb{R}$, we have, \begin{align*} \mathbb{E}[\exp(\lambda(Y - \mathbb{E}[Y]))] & = \int_{-\infty}^{\infty} \exp(\lambda(y - \mathbb{E}[Y])) f_Y(y) \ \mathrm{d}y \\ & = \int_{-\infty}^{\infty} \exp(\lambda(y - \mathbb{E})) \frac{f_X(y)}{\Pr(|X| \leq R+B)} \cdot \mathbb{I}{[-(R+B), (R+B)]} \ \mathrm{d}y \\ & \leq \exp(\lambda(\mu - \mathbb{E}[Y])) \int_{-\infty}^{\infty} \exp(\lambda(y - \mu)) \frac{f_X(y)}{\Pr(|X| \leq R+B)} \ \mathrm{d}y \\ & \leq \frac{\exp(\lambda(\mu - \mathbb{E}[Y]))}{\Pr(|X| \leq R+B)} \mathbb{E}[\exp(\lambda(X - \mu))]\\ & \leq \frac{\exp(\lambda(\mu - \mathbb{E}[Y]) + \lambda^2 \sigma^2/2)}{\Pr(|X| \leq R+B)}. \end{align*}

Thus, for any $\lambda > 0$ \begin{align*} \Pr(Y - \mathbb{E}[Y] > t + t_0) & \leq \mathbb{E}[\exp(\lambda(Y - \mathbb{E}[Y]))] \cdot \exp(-\lambda(t + t_0)) \\ & \leq \frac{1}{\Pr(|X| \leq R+B)} \cdot \exp \left(\lambda(\mu - \mathbb{E}[Y]) + \frac{\lambda^2 \sigma^2}{2} -\lambda(t + t_0) \right) \\ & \leq \frac{1}{\Pr(|X| \leq R+B)} \cdot \exp \left(\frac{\lambda^2 \sigma^2}{2} -\lambda t \right) \\ & \leq \frac{1}{\Pr(|X| \leq R+B)} \cdot \exp \left(-\frac{t^2}{2\sigma^2} \right). \end{align*} We can use a similar argument to establish $\Pr(Y - \mathbb{E}[Y] < -(t + t_0))$ and conclude that $\displaystyle \Pr(|Y - \mathbb{E}[Y]| > (t + t_0)) \leq \frac{2}{\Pr(|X| \leq R+B)} \cdot \exp \left(-\frac{t^2}{2\sigma^2} \right)$.

It is straightforward to extend this argument to any interval $[a,b]$ and also you can simplify the denominator and express it in terms of $R$.