Maximum Likelihood Estimate of Laplacian distribution with one parameter $\sigma$

828 Views Asked by At

Given variant of laplacian distribution

$p(x|\sigma) = \frac{1}{2 \sigma} e^{\frac{|x|}{\sigma}}, \sigma > 0$ with parameter $\sigma$

I got MLE for $\sigma$ as $\frac{\sum_{i=1}^n |x_i|}{n}$

calculating

$E[|x|] = \int_{-\infty}^{\infty} \frac{1}{2\sigma} |x| e^{\frac{-|x|}{\sigma}} dx \\ = \int_{0}^{\infty} \frac{1}{\sigma} |x| e^{\frac{-|x|}{\sigma}} dx = \sigma$

$E[|x|^2] = \int_{-\infty}^{\infty} \frac{1}{2\sigma} |x|^2 e^{\frac{-|x|}{\sigma}} dx \\ = \int_{0}^{\infty} \frac{1}{\sigma} |x|^2 e^{\frac{-|x|}{\sigma}} dx = 2\sigma^2$

$\text {var} (|x_i|) = E[|x_i|^2] - (E[|x|])^2 = 2\sigma^2 - \sigma^2 = \sigma^2$

Variance($\hat \sigma$) = variance ($ \frac{\sum_{i=1}^n |x_i|}{n}$) =\frac{1}{n} $Var(|x_i|) =\frac{1}{n} \sigma^2$

Bias($\hat \sigma$) = $E[\hat \sigma] - \sigma = E[\frac{\sum_{i=1}^n |x_i|}{n}] - \sigma = (1/n)n \sigma = \sigma $

Now MSE($\hat \sigma$) = $Bias^2 + \text {variance} = \frac{1}{n} \sigma^2 $

May i know is this correct?

1

There are 1 best solutions below

6
On BEST ANSWER

The bias is not correct. Notice that unless every $X_i = 0$, the sum $$\hat \sigma = \frac{1}{n} \sum_{i=1}^n |X_i| > 0.$$ So the calculation of the expectation is not $0$, because the estimator $\hat \sigma$ is almost surely strictly positive.

In order to compute the bias, you must compute the expectation of the absolute value of a single observation; i.e., $$\operatorname{E}[|X|] = \int_{x=-\infty}^\infty \frac{|x|}{2\sigma} e^{-|x|/\sigma} \, dx = \int_{x=0}^\infty \frac{x}{\sigma} e^{-x/\sigma} \, dx.$$ Then use the linearity of expectation to compute $\operatorname{E}[\hat \sigma]$.

To compute the variance, also begin from a single observation: $$\operatorname{E}[|X|^2] = \int_{x=-\infty}^\infty \frac{|x|^2}{2\sigma} e^{-|x|/\sigma} \, dx.$$ Then $$\operatorname{Var}[|X|] = \operatorname{E}[|X|^2] - \operatorname{E}[|X|]^2,$$ and then, by independence of the individual observations, $$\operatorname{Var}[\hat \sigma] = \frac{1}{n^2} \sum_{i=1}^n \operatorname{Var}[|X_i|].$$