$E(X \mid X > x)$ is increasing in $x$. Why?

Question

For two points $x < x'$ and a random variable $X$, we must have $E(X\mid X > x )\leq E(X\mid X > x' )$. This is "obviously" true because the center of the truncated distribution shifts to the right. How do I prove that?

I tried working with an iid copy $X^*$ of $X$ to show that the expectation of $X1(X>x)1(X^*>x')$ is smaller than the expectation of $X1(X>x')1(X^*>x)$ but I'm not having any luck with that.

All results I can find either focus on normality or assume densities.

Answer 1

Let $Y$ be an iid copy of $X$.

Notice the following inequality holds $$(X-Y)(1_{X>x'}1_{Y>x}-1_{Y>x'}1_{X>x})\geq 0$$

and take expectations to find $$E(X1_{X>x'})P(Y>x)-E(X1_{X>x})P(Y>x')-E(Y1_{Y>x})P(X>x')+E(Y1_{Y>x'})P(X>x)\geq 0,$$

which rewrites $$2E(X1_{X>x'})P(X>x)-2E(X1_{X>x})P(X>x')\geq 0,$$ thus

$$\frac {E(X1_{X>x'})}{P(X>x')}\geq \frac {E(X1_{X>x})}{P(X>x)}$$

Answer 2

For continuous random variables only:

Let $f(t) = E[X | X > t ] = \frac{ \int_t^\infty xf_X(x) dx}{\int_t^\infty f_X(x) dx}$

Define $g(t) = \int_t^\infty xf_X(x) dx$ and $h(t) = \int_t^\infty f_X(x) dx$ such that $f(t) = \frac{g(t)}{h(t)}$.

$g(t)$ and $h(t)$ are differentiable $\Rightarrow f(t)$ is differentiable by Quotient Rule.

$$ \begin{align*} f'(t) &= \frac{-tf_X(t) \int_t^\infty f_X(x)dx + f_X(t)\int_t^\infty xf_X(x)dx}{h(t)^2} \\ \Rightarrow f'(t) h(t)^2 &= f_X(t) \bigg( \int_t^\infty (x - t)f_X(x)dx \bigg) \geq 0\\ \end{align*} $$

Hence $f$ is an increasing function!

Answer 3

Let $Y = X \mid X>x$. Then $Y \mid Y > x'$ is the same as $X \mid X > x'$, so it's enough to show that $\mathbb E[Y] \le \mathbb E[Y \mid Y > x']$: in other words, conditioning on $Y$ being high increases the expectation of $Y$.

For this, we have the law of total expectation: $$ \mathbb E[Y] = \mathbb E[Y \mid Y > x'] \Pr[Y > x'] + \mathbb E[Y \mid Y \le x'] \Pr[Y \le x']. $$ In other words, $\mathbb E[Y]$ is a weighted average of $\mathbb E[Y \mid Y > x']$ and $\mathbb E[Y \mid Y \le x']$.

There are two cases:

Case 1. $\mathbb E[Y] \le x'$. In this case, we have $\mathbb E[Y \mid Y > x'] \ge \mathbb E[Y] $ because because $Y \mid Y > x'$ is always bigger than $\mathbb E[Y]$.

Case 2. $\mathbb E[Y] > x'$. In this case, we always have $\mathbb E[Y \mid Y \le x'] \le \mathbb E[Y]$, because $Y \mid Y \le x'$ is always less than $\mathbb E[Y]$. To have the weighted average come out to $\mathbb E[Y]$, we must have $\mathbb E[Y \mid Y > x'] \ge \mathbb E[Y] $ to compensate.

In both cases, we get $\mathbb E[Y \mid Y > x'] \ge \mathbb E[Y] $.

Answer 4

Non-only is the function non-decreasing in $x$, but its derivative is explicit.

Assume $X$ is non-negative for simplicity and let $F(x)=P(X<x)$ the cdf. As the CDF is monotone, the monotone differentiation theorem given in Theorem 53 of https://terrytao.wordpress.com/2010/10/16/245a-notes-5-differentiation-theorems/ gives that $F(x)$ is differentiable almost everywhere. The same goes for $G:x\mapsto \int_x^\infty P(X>t)dt$, it is monotone decreasing and thus differentiable everywhere with derivative equal to $P(X>x)$.

The identity holds $E[X|X>x] = x + (1-F(x))^{-1}\int_x^\infty P(X>t)dt$. Before giving a proof, let's explain why this shows monotonicity of $h(x) = E[X|X>x]$. If the cdf $F$ and $G$ are both diffentiable at $x$, then $h$ is also differentiable at $x$ as elementary addition/product/inverse of differentiable functions at by the product rule $h'(x) = 1 - 1 + F'(x)(1-F(x))^{-2} \int_x^\infty P(x>t)dt \ge 0$. Hence almost everywhere, the derivative of $h$ exists and is non-negative.

It remains to study the points at which $F$ is not differentiable: we can proceed ``guided'' by the differentiable case: for any $a>0$ since $(1-F(x+a))^{-1} \ge (1-F(x))^{-1}$ \begin{align} E[X|X>x+a] - E[X|X>x] &\ge (x+a) + (1-F(x))^{-1}\int_{x+a}^\infty P(X>t)dt - x - (1-F(x))^{-1}\int_{x}^\infty P(X>t)dt \\&= a - (1-F(x))^{-1} \int_x^{x+a} P(X>t)dt \\&\ge a - (x+a - x) P(X>x) \ge 0 \end{align} thanks to $-P(X>t)\ge -P(X>x)$ for all $t\in[x,x+a]$ for the last inequality.

Why is the identity $h(x) = E[X|X>x] = x + \int_x^\infty P(X>t)dt$ true? It's a consequence of the well known identity $E[X]=\int_0^\infty P(X>t)dt$ for non-negative $X$ which follows from Fubini's theorem. Here, \begin{align} P(X>x)x + \int_x^\infty P(X>t)dt &= P(X>x)\int_0^x 1 dt + \int_x^\infty P(X>t)dt \\&= \int_0^\infty P(X> \max\{x,t\})dt \\&= E[I\{X>x\}X] \end{align} and the desired formula for $E[X|X>x]=E[I\{X>x\}X]/P(X>x)$ follows.