Maximizing expectation integral given PDF inequality

Question

I came across a paper that in the context of one of its proofs takes an upper bound on a random variable's probability distribution and turns it into a bound on its expected value.

In particular, let $X$ be a random variable with PDF $f$. The result is that if $$P(X \geq x) = \int_x^\infty f(\gamma) d\gamma \leq 2 e^{-2mx^2}$$ then $$E\left[e^{2(m-1)X^2}\right] = \int_0^\infty f(\gamma)e^{2(m-1)\gamma^2} d\gamma \leq 4m.$$

The paper proposes a proof saying that the first constraint is met with equality for $g(\gamma) = 8 m \gamma e^{-2m\gamma^2}$ and that this distribution will maximize the expected value expression. For this distribution one can compute the expected value to be $E\left[e^{2(m-1)X^2}\right] = \int_0^\infty 8 m \gamma e^{-2m\gamma^2}e^{2(m-1)\gamma^2} d\gamma = 4m$.

I believe that the result is true (I came up with a different proof) but I am confused about the validity of this argument. Why is it true that a distribution, $g$, that achieves equality in the first constraint will also maximize the expected value expression?

If it was true that for such a $g$ we knew $g(\gamma) \geq f(\gamma)$ for any $f$ satisfying the first inequality then I could see why this argument is valid. However, I don't think that's necessarily true. Consider the following counterexample. Suppose that $\int_x^1 f(\gamma) d\gamma \leq 1 - x$. Equality can be achieved for $g(\gamma) = 1$. However, $f(\gamma) = 3/2 e^{-\gamma}$ also meets the inequality and yet it is not true that $ 3/2 e^{-\gamma} \leq 1$ for all $\gamma$.

So I am wondering if this argument is valid and if so how?

Answer 1

I believe that the argument from the paper is valid. Thanks to @BGM's explanation and the reference he gave I now see how to justify it. Just reposting the argument here a bit more explicitly since the notation and vocabulary from @BGM's reference are a little bit different and to me the connection wasn't obvious.

The constraint we are given is that $$ \int_x^\infty f(\gamma) d\gamma \leq \int_x^\infty g(\gamma) d\gamma = e^{-2mx^2} \quad \forall x>0 $$ and we want to show that $$ \int_0^\infty u(\gamma) f(\gamma) d\gamma \leq \int_0^\infty u(\gamma) g(\gamma) d\gamma $$ where $u(\gamma) = e^{2(m-1)\gamma^2}.$

Equivalently, if we assume that $X$ and $Y$ are random variables with support $(0, \infty)$ and $X$ has distribution $f$ and $Y$ has distribution $g$ we want to show $E[u(X)] \leq E[u(Y)]$.

The first inequality is equivalent to an inequality of the CDF of $X$ and $Y$, $F$ and $G$, respectively. Specifically, we have $F(x) \geq G(x)$ since $$ F(x) = 1 - \int_x^\infty f(\gamma) d\gamma \geq 1 - \int_x^\infty g(\gamma) d\gamma = G(x) \quad \forall x > 0. $$

We note that because $F$ and $G$ are increasing functions with $F(0) = G(0) = 0$ and $F(x) = G(x) = 1$ as $x \to \infty$ we have $F^{-1}(p) \leq G^{-1}(p)$ and as CDFs their derivatives are the distributions, i.e. $dF/d\gamma = f(\gamma)$ and $dG/d\gamma = g(\gamma)$.

The desired result now follows from a $u$-substitution $p=F(\gamma)$ and a $u$-substition back, $p = G(\gamma)$: $$ \int_0^\infty u(\gamma) f(\gamma) d\gamma = \int_0^1 u(\gamma) f(\gamma) \frac{dp}{f(\gamma)} = \int_0^1 u(F^{-1}(p)) dp \leq \int_0^1 u(G^{-1}(p)) dp = \int_0^\infty u(\gamma)g(\gamma) d\gamma $$ where the inequality follows because $u$ is an increasing function.

In summary, this says that if we have an upper bound on the survival function of a random variable we can use a distribution which achieves equality with this bound to set a bound on the expected value of any increasing function of the random variable.

Answer 2

$\def\e{\mathrm{e}}\def\d{\mathrm{d}}$The argument used in the paper is indeed wrong. Here is a proof using integration by parts.

For any $f$ satisfying$$ \int_x^{+\infty} f(u) \,\d u \leqslant 2\e^{-2mx^2}, \quad \forall x > 0 $$ using Tonelli's theorem,\begin{align*} &\mathrel{\phantom{=}}{} \int_0^{+\infty} f(u) \e^{2(m - 1)u} \,\d u = \int_0^{+\infty} f(u) \left(1 + 2(m - 1) \int_0^u \e^{2(m - 1)v} \,\d v\right) \d u \\ &= \int_0^{+\infty} f(u) \,\d u + 2(m - 1) \int_0^{+\infty} \int_0^u f(u) \e^{2(m - 1)v} \,\d v \d u\\ &= 1 + 2(m - 1) \int_0^{+\infty} \int_v^{+\infty} f(u) \e^{2(m - 1)v} \,\d v \d u\\ &= 1 + 2(m - 1) \int_0^{+\infty} \e^{2(m - 1)v} \,\d v \int_v^{+\infty} f(u) \,\d u\\ &\stackrel{(1)}{\leqslant} 1 + 2(m - 1) \int_0^{+\infty} \e^{2(m - 1)v} \,\d v \int_v^{+\infty} g(u) \,\d u\\ &= \int_0^{+\infty} g(u) \e^{2(m - 1)u} \,\d u = 4m. \end{align*}

To achieve equality, from (1) there is$$ \int_x^{+\infty} f(u) \,\d u = \int_x^{+\infty} g(u) \,\d u, \quad \forall x > 0 $$ i.e.$$ \int_x^{+\infty} (f(u) - g(u)) \,\d u = 0, \quad \forall x > 0 $$ thus $f = g\ \text{a.e.}$

Answer 3

As requested I repost the argument here. Let $X^*$ be a continuous random variable with pdf $$ f_{X^*}(x) = 8mxe^{-2mx^2}, m > 0$$

The support is not specified so I guess the author is only concerned about the tail behavior. Assume the support to be $[\underline{x}, +\infty)$. The survival function is given by

$$ \Pr\{X^* \geq x\} = \int^{+\infty}_x 8mue^{-2mu^2}du = 2e^{-2mx^2}, x \geq \underline{x}$$

and we can compute $\underline{x}$ anyway (not important) $$ 2e^{-2m\underline{x}^2} = 1 \Rightarrow \underline{x} = \underline{x} = \sqrt{ \frac {\ln 2} {2m}}$$

Define

$$ \mathcal{X}=\left\{X: \Pr\{X \geq x\} \leq 2e^{-2mx^2} ~\forall x \geq \underline{x}, m > 0, \Pr\{X \geq 0\} = 1 \right\}$$

be the collection of random variables satisfying the given inequality constraint. Then from the above calculation $X^* \in \mathcal{X}$ and $$ \Pr\{X^* \geq x\} \geq \Pr\{X \geq x\} ~~ \forall x \in \mathbb{R}, X \in \mathcal{X} $$

And this is the usual definition of $X^*$ first-ordered stochastic dominate $X$ (for every $X \in \mathcal{X}$). From the link https://ocw.mit.edu/courses/economics/14-123-microeconomic-theory-iii-spring-2015/lecture-notes-and-slides/MIT14_123S15_Chap4.pdf P.1-3, it explained this term and proved that for every weakly increasing function $u$, we also have

$$ E[u(X^*)] \geq E[u(X)] ~~ \forall X \in \mathcal{X}$$

and this is another equivalent definition from above. The proof is written there and is quite clear so I do not think it is necessary to retype line by line. If there is any doubt then we can discuss afterward.

The final thing is to check the given function $h(x) = e^{2(m-1)x^2}$ is weakly increasing (on $[\underline{x}, +\infty)$)- it merely require $m > 1$.