Expected value of the smaller of two random variables.

Question

I am studying machine learning by Kevin Murphy and a very simple problem got me stuck.

Note for the jaintors - I know a solution is probably available, but I want to know why my solution is bad.

I will rewrite the problem here so I can explain what is wrong with my solution.

Suppose X, Y are two points sampled independently at random from the interval $[0,1]$. What is the expected location of the leftmost point?

Here is a picture of my approach:

I have split a $[0,1]\times [0,1]$ plane into two triangles, one of which satisfies the condition $Y<X$. This happens with probability of $0.5$, but I can, without loss of generality, just calculate the result for this triangle, since for the other one the result will be the same, just with flipped variables.

This way, I already have $Y<X$. Now, I need to derive a pdf for $Y$, given $X=x$. If $X=x$, than $Y$ must be smaller than $x$. This means, that $y\in(0,x)$.

Going further, if $y \in (0,x)$, and $y$ is uniformly distributed, than the pdf of $Y$ given $X$ must be:

$P(Y|X=x)={1\over x}$ - since the pdf must integrate to one.

Now what I need is the pdf of $Y$ itself, so I can calculate it's expected value. For this reason, I sum over the whole available $X$ space:

$p(y)=\int_{y}^{1}{1 \over x}dx$ - i integrate from $y$, because $x > y$.

This way, I get $p(y) = -\log(y)$.

Then, the expected value is: $\mathbb{E}[y] = -\int_{0}^{1}y\log(y) = {1 \over 4}$.

However, using another approach, I can do:
$p(\min(X,Y) > x) = p(X > x, Y > x) = (1 - x)(1 - x) = (1-x)^2$
Let $Z = \min(X, Y)$, then:
$p(Z > x) = (1-x)^2$, and:
$p(Z < x) = 1 - (1-x)^2$, giving:
$P_Z(z) = 1-(1-z)^2$, which is the cdf of $Z$ - so I can obtain the $pdf$ by differentiation.

$p(z) = {d \over {dz}}P_Z(z) = 2(1-z)$ Now, I can go straight into the expected value:
$\mathbb{E}[z] = \int_{0}^{1}2(z-z^2) = {1 \over 3}$

After checking on the internet, the second result i got (1/3), is the valid one. But I'm not really that concerned with why this is the right result, but rather why my first approach was wrong.

Answer 1

Indeed $f_{Y\mid X, Y<X}(y\mid x) = x^{-1}\mathbf 1_{0\leqslant y\leqslant x\leqslant 1}$ as per your argument from symmetry.

However, your first approach then makes on the erroneous claim: $$f_{Y\mid Y<X}(y)=\int_y^1 f_X(x)f_{Y\mid X, Y<X}(y\mid x)\,\mathrm d x \tag{false}$$

However, properly applying the Law of Total Probability gives: $$f_{Y\mid Y<X}(y)=\int_y^1 f_{X\mid X<Y}(x)\,f_{Y\mid X, Y<X}(y\mid x)\,\mathrm d x\tag{true}$$

And $f_{X\mid X<Y}(x)$ does not equal $f_X(x)$.   If it did, the same argument would apply to $f_{Y\mid Y<X}(y)$ equaling $f_Y(y)$ (contrary to your result).

---

As an alternative to the second approach, just apply the definition of conditional probability measures, and the independence of $X,Y$:$$\begin{align}f_{Y\mid Y<X}(y) &=\dfrac{\mathsf P(Y<X\mid Y=y)\, f_Y(y)}{\mathsf P(Y<X)}\\[1ex]&=2~(1-y)~\mathbf 1_{0\leqslant y\leqslant 1}\end{align}$$

And so: $$\begin{align}\mathsf E(Y\mid Y<X) & = \int_0^1 2y(1-y)\mathrm d y \\[1ex] &= 1/3\end{align}$$

$\blacksquare$

Answer 2

In your first approach, the conditional distribution of $Y$, given $X=x$, is uniform on $(0,x)$; therefore $E[Y\mid X=x]=x/2$. Integrating out $y$ in the joint density of $(X,Y)$ you get the marginal density $f_X(x)=2x$, $0<x<1$. The mean of $Y$ is therefore $$ E[Y] =\int_0^1 E[Y\mid X=x]f_X(x) dx =\int_0^1 {x\over 2}2x dx=\int_0^1 x^2 dx = {1\over 3}. $$

Alternative: Let $L$ be the leftmost of $X$ and $Y$. You have $L>x$ if and only if both $X>x$ and $Y>x$; thus $P[L>x] =(1-x)^2$, $0<x<1$. And so $$ E[L] =\int_0^1 P[L>x] dx =\int_0^1 (1-x)^2 dx =-{(1-x)^3\over 3}\Big|_{0}^1 ={1\over 3}. $$