Show that the mle of $\theta$ is the median of a sample

Question

The double exponential distribution is $$f(x|\theta) = \frac{1}{2}e^{-|x-\theta|},\ \ \ \ \ \ -\infty < x < \infty$$ We are looking at an i.i.d. sample of size $n = 2m + 1$. I would like to show that the maximum likelihood of $\theta$ is the median of the sample. Any ideas on how to do this?

Answer 1

Suppose that your values are stored from the lowest to the highest, i.e. $$ x_1 \leq x_2 \leq ... \leq x_{m-1} \leq x_m \leq x_{m+1} ...\leq x_{2m+1} $$

the likelihood is given by: $$ L(x_1,...,x_{2m+1},\theta) = f(x_1,\theta)...f(x_{2m+1},\theta) = \frac{1}{2^{2m+1}} \exp{\left(- \sum_{k=1}^{2m+1} \vert x_k - \theta \vert \right)}$$

Therefore, the log-likelihood is:

$$ l(x_1,...,x_{2m+1},\theta) = \ln( L(x_1,...,x_{2m+1},\theta) ) = - (2m+1)\ln2 - \sum_{k=1}^{2m+1} \vert x_k - \theta \vert $$

You want the value of $\theta$ that maximizes the likelihood, i.e. the value of $\theta$ that minimizes the sum $g(\theta) = \sum_{k=1}^{2m+1} \vert x_k - \theta \vert$.

We want to get rid of the absolute values because we know they are not easy to differentiate. We now that $\vert x_k - \theta \vert = x_k - \theta$ if $\theta \leq x_k $, and $\vert x_k - \theta \vert =\theta - x_k$ if $\theta \geq x_k $.

We know that $$ x_1 \leq x_2 \leq ... \leq x_{m-1} \leq x_m \leq x_{m+1} ...\leq x_{2m+1} $$

so we know that $\theta$ is necessarily:

a) either in the interval $]-\infty, x_1]$;

b)either in the interval $[x_1,x_2]$;

c) either in the interval $[x_2,x_3]$;

etc

y) either in the interval $[x_{2m},x_{2m+1}]$;

z) either in the interval $[x_{2m+1},+\infty[$.

So basically you have to study $g(\theta)$ on each of these intervals, and find where the minimal value of $g(\theta)$ is reached.

Let's begin by the first and the last case: if $\theta \in ]-\infty, x_1]$, we have $\theta \leq x_k$ for all $k$, because the values are stored from the lowest to the highest. Therefore $\vert x_k - \theta \vert = x_k - \theta$ for all $k$, and

$$ g(\theta) =\sum_{k=1}^{2m+1} ( x_k - \theta ) = \sum_{k=1}^{2m+1} x_k - (2m+1) \theta$$

clearly, this function is linear with a negative slope $-(2m+1)$. Therefore, it is decreasing on the interval $\theta \in ]-\infty, x_1]$. On this interval, $g(\theta)$ is minimal for $\theta = x_1$.

Conversely, on the interval $[x_{2m+1},+\infty[$ you will find that $g(\theta)$ is linear with a positive slope, and therefore is minimal on this interval for $\theta = x_{2m+1}$.

Now, we are stuck with the other intervals. Suppose that we study $g(\theta)$ for $\theta \in [x_j,x_{j+1}]$, where $j$ is an integer between $1$ and $2m$.

We know that $x_j \leq \theta \leq x_{j+1}$, therefore $\theta \geq x_k$ for all $k \in \{1,...,j\}$, and $\theta \leq x_k$ for all $k \in \{j+1,...,2m+1\}$. This allows us to get rid of the absolute values:

$$ g(\theta) =\sum_{k=1}^{2m+1} \vert x_k - \theta \vert = \sum_{k=1}^{j}(\theta - x_k) + \sum_{k=j+1}^{2m+1}( x_k - \theta) = (2j-(2m+1))\theta + ... $$

where "$...$" is a constant term that does not depend on $\theta$. So, once again, your function is linear with a slope equal to $2j-(2m+1)$, which is negative if $j \in \{1,...m\}$ and positive if $j \in \{m+1,...2m\}$.

So $g(\theta)$ is decreasing on $[x_1,x_2] ,...,[x_m,x_{m+1}]$ and then increasing on $[x_{m+1},x_{m+2}], ... [x_{2m},x_{2m+1}]$.

Here we are! We have shown that the function is decreasing on all the intervals until $x_{m+1}$, and increasing on all the intervals after. Therefore, it is minimal for $\theta = x_{m+1}$, which is the median value of your sample.