Minimizing $\mathbb{E}|X-r|$ and $\mathbb{E}|X-r|^2$ in terms of $r$

Question

I came across this problem in Shiryaev's book "Probability":
1. For what values of $r\in\mathbb{R}$, $\;\mathbb{E}|X-r|$ is minimal, if known that $X$ is a random variable, so that $\mathbb{E}|X|<\infty$?
2. For what values of $r\in\mathbb{R}$, $\;\mathbb{E}|X-r|^2$ is minimal, if known that $X$ is a random variable, so that $\mathbb{E}X^2<\infty$?
It is obviously related to Markov and Chebyshev inequalities: $\mathbb{P}(|X|\geq r) \leq \frac{\mathbb{E}|X|}{r}$, but it is not clear to me, when does the equality take place (as we are looking for minimum of $\mathbb{E}$).
*EDIT: $X$ is supposed to be a discrete r.v.

Answer 1

The following solution to part 1 assumes that $X$ is a continuous r.v.; if you cannot make this assumption then use this as a guide towards a more general solution. Many of the integrals below assume that $\mathbb E|X| < \infty$ for convergence, which is given in the problem. $f_X$ and $F_X$ are the pdf and cdf respectively. We note that \begin{align*} \mathbb E |X-r| &= \int_{-\infty}^r f_X(x) (r-x) \, dx + \int_r^{\infty} f_X(x) (x-r) \, dx \\ &= \int_{-\infty}^{\infty} f_X(x)(x - r) \, dx - 2 \int_{-\infty}^{r} f_X(x)(r - x) \, dx \\ &= \left(\mathbb E[X] - r\right) - 2 \left(F_X(r)r - \int_{-\infty}^{r} f_X(x) x \, dx\right). \end{align*} By IBP, $$\int_{-\infty}^{r} f_X(x)x \, dx = F_X(r)r - \int_{-\infty}^{r} F_X(x) \, dx.$$ So we have $$\mathbb E |X-r| = \mathbb E [X] - \left(2 \int_{-\infty}^{r} F_X(x) \, dx + r\right).$$ We want to minimise this in $r$. Because of continuity, it will take a minimum as we approach either $-\infty$ or $\infty$ (neither of which is the case, clearly), or at a point where $\frac{\partial}{\partial r} = 0$. Differentiating we obtain $$\frac{\partial}{\partial r} \mathbb E|X - r| = 1 - 2 F_X(r),$$ which will be zero when $F_X(r) = \frac 1 2$. Since the range of such solutions in $r$ is an interval (and nonempty, by the assumption of continuity), all of them minimise the expression. In other words, the expression is minimised when $r$ is a median of the data set. In more general cases, the optimal $r$ is still the median, but I'll leave this to you to prove.

For the second part, just note that $$\mathbb E \left[(X-r)^2\right] = \mathbb E[X^2] - 2 r \mathbb E[X] + r^2,$$ which is easy to minimise. This doesn't use the continuity assumption from before.

Answer 2

For the first part, Let $T$ be a value such that $$ \mathbb P(X > T)\le 1/2, \qquad \mathbb P(X < T)\le 1/2 $$ It always exists, and it is called the median of $X$. We can prove that $r=T$ minimizes $\mathbb E|X-r|$. In fact, if $r>T$, then $$ \mathbb E|X-r| = \int_{X\ge r} X-r + \int_{r > X\ge T} r-X + \int_{T > X} r-X\\ \mathbb E|X-T| = \int_{X\ge r} X-T + \int_{r > X\ge T} X-T + \int_{T > X} T-X\\ \mathbb E|X-r| - \mathbb E|X-T| = \int_{X\ge r} T -r + \int_{r > X\ge T} r-2X+T + \int_{T > X} r-T \\ \ge \int_{X\ge r} T -r + \int_{r > X\ge T} T-r +\int_{T > X} r-T\\ = \int_{X\ge T} T-r +\int_{T > X} r-T\\ \ge \frac 12(T-r) + \frac 12(r-T) =0 $$