ML estimator of an double exponential distribution

Question

Im trying to figure out the ML estimator of $$f_X(x)=\frac{1}{2\beta}\exp\left(-\frac{|x|}{\beta}\right)$$ as well as the variance of this estimator. So far I have $$L(\beta;x)=\prod_{i=1}^n\frac{1}{2\beta}\exp\left(-\frac{|x_i|}{\beta}\right),$$ $$\ln[L(\beta;x)]=\sum_{i=1}^n\left[-\ln(2\beta)-\frac{|x_i|}{\beta}\right] =-n\ln(2\beta)-\frac{1}{\beta}\sum_{i=1}^n|x_i|,$$ $$\frac{\partial \ln[L(\beta;x)]}{\partial\beta}=-n\beta+\frac{1}{\beta^2}\sum_{i=1}^n|x_i|=0.$$ So I get $$\hat\beta=\frac{1}{n}\sum_{i=1}^n|X|$$ and with a hint in my textbook $\text{Var}(|X|)=\beta^2$ $$\text{Var}(\hat\beta)=\frac{1}{n^2}\sum_{i=1}^n \text{Var}(|X|)=\frac{\beta^2}{n}$$ Unfortunately this variance does not correspond with the variance in my distribution summary for an double exponential distribution, which is $2\beta^2$.

I think my mistake arises from the absolute value. I hope someone can figure it out. Thanks!

Answer 1

You have shown that $\hat \beta = \frac{1}{n}\sum_{i=1}^n |X_i|$ is the MLE of $\beta$ in a Laplace (double-exponential) distribution with median $0$ and scale parameter $\beta$ according to the given PDF. (See Wikipedia on 'Laplace distribution'.)

One way to generate $X,$ having this distribution with $\beta = 1,$ is to let $X = Y_1 - Y_2,$ where $Y_i$ are independently exponential with rate 1. In R statistical software below I generate $m = 100,000$ samples of size $n = 50$ from this distribution, find $\hat \beta$ for each sample, and simulate its expectation $E(\hat \beta)$ by averaging these 100,000 values of $\hat \beta.$ The result is consistent with $E(\hat \beta) = 1.$ As you remarked above, the variance of this Laplace distribution is $\sigma^2 = 2\beta^2 = 2.$ With $m = 100,000$ samples, results should be accurate to two (maybe three) decimal places. [For comparison, I also found the medians of the $m$ samples, which are consistent with population median $0,$ and their variances are consistent with $\sigma^2 = 2.$ Finally, the variance of the 100,000 $\hat \beta$s is consistent with $Var(\hat \beta) = \beta^2/n = 1/50.$]

m = 10^5;  n = 50;  x = rexp(m*n) - rexp(m*n)
DTA = matrix(x, nrow=m)    # m x n matrix, each row a sample of n
h = apply(DTA, 1, median)  # vector of m sample medians
b = rowMeans(abs(DTA))     # vector of m beta-hats
v = apply(DTA, 1, var)     # vector of m sample variances
mean(h)
## -9.330546e-05           # consistent with pop median = 0
mean(b)
## 0.9999066               # consistent with pop beta = 1
mean(v)
## 1.998009                # consistent with pop var = 2
var(b);  1/n
## 0.02005612              # consistent with Var(beta hat) = 1/50
## 0.02

I hope this demonstration helps you identify the role of the estimator $\hat \beta$ and its relationship to the population variance.

Below is are plots of original Laplace data x, sample medians h, sample $\hat \beta$s b, and sample variances v.