Difference in expected values in Mathematica?

Question

Could someone explain why I am getting different outputs when I calculate the following two expressions on Mathematica?

1. First:

 {
  Expectation[x - (c) \[Conditioned] x > (c), 
   x \[Distributed] UniformDistribution[{0, 1}]]
  }]

2. Second:

Assuming[0 < c && c < 1,
 {
  Integrate[(x - c) Boole[x > c], {x, c, 1}]
  }]

What is the right way to calculate the expected value of $x-c$ conditional on $x>c$? and what is the intuition behind the first and the second one?

Answer 1

A conditional expected value is an ordinary expected value, calculated for the conditional PDF. We will have $$\mathbb E[X-c \mid X>c] = \int_c^\infty (x-c)f_{X\mid X>c}(x)\,dx.$$ Two things need to be done to turn the PDF $f_X(x)$ into the conditional PDF $f_{X \mid X>c}(x)$:

1. Zero out the PDF where $x \le c$.
2. Rescale so that it integrates to $1$.

In the case of the uniform, $f_X(x)$ is $1$ on $[0,1]$ and $0$ otherwise. You have done the first step: zero out the PDF. (You have actually done it twice: setting the bounds of the integral from $c$ to $1$ and multiplying by Boole[x>c] $ = \mathbf 1_{x>c}$ are overkill. Either one would have been enough on its own.)

But the function that is equal to $1$ on $[c,1]$ and $0$ otherwise cannot be the conditional PDF $f_{X \mid X>c}(x)$, because it is not a PDF: its integral is not $1$. We must rescale: divide it by $1-c$. A more general rule is $$f_{X \mid X>c}(x) = \frac{f_X(x) \mathbf 1_{x>c}}{\int_c^\infty f_X(x)\,dx}.$$ (You can adapt this rule to conditioning on other events about $X$ in the same way.)

With this PDF, we can now correctly compute $$\mathbb E[X-c \mid X>c] = \int_c^1 \frac{x-c}{1-c}\,dx.$$