Find the $P(\theta - 0.1 \leq MLE \leq \theta + 0.1)$

Question

I have a continuous random variable with density: $f(x|\theta) = \frac{\theta}{x^{\theta+1}}$.

I have calculated the MLE($\hat\theta$) as: $\frac{n}{\sum_{i}^{n}log(x_i)}$

I am stuck on figuring out how to estimate $P(\theta - 0.1 \leq \hat\theta \leq \theta + 0.1)$?

Thank you.

Answer 1

You are given the Pareto density $$f(x\mid \theta)=\frac{\theta}{x^{\theta+1}}\mathbf1_{x>1}\quad,\,\theta>0$$

This is a one-parameter exponential family, so we can be certain that the MLE of $\theta$ has an asymptotic normal distribution.

For large $n$, you can approximate the probability by using the fact that

$$\sqrt{n}(\hat\theta-\theta)\stackrel{a}\sim N\left(0,\frac{1}{I_{X_1}(\theta)}\right)$$, where $I_{X_1}(\theta)=E_{\theta}\left[\frac{\partial}{\partial\theta}\ln f(X_1\mid\theta)\right]^2$ is the information in a single observation.

This looks like what you are asked to do, as opposed to finding the exact probability.

Since the distribution function of $X$ is $F(x\mid\theta)=1-x^{-\theta}$ for $x>1$, we have $1-X^{-\theta}\sim U(0,1)$, or equivalently, $X^{-\theta}\sim U(0,1)$. Therefore, $-2\ln X^{-\theta}=2\theta\ln X\sim\chi^2_2$.

So if $X_1,X_2,\ldots,X_n$ are i.i.d having pdf $f$,

$$T=2\theta\sum_{i=1}^n \ln X_i\sim \chi^2_{2n}$$

Since $\hat\theta=\frac{2n\theta}{T}$, you can also use the distribution of $T$ to find the probability in terms of chi-square fractiles.

Answer 2

So I will tell you the step by step process to do this, and give you some hints too.

$1.$ Find the distribution of $\hat{\theta} - \theta$. As a random variable, $\hat{\theta} = \frac{n}{\sum_{i}^n \log(X_i)}$, with $X_i \sim f(x|\theta)$

$2.$ We ask what is the probability of $P(-0.1 \leq \hat{\theta}-\theta \leq 0.1)$.

Once we know the distribution of $\hat{\theta}-\theta$ we will apply some adjustment so that we can use statistical tables to look this up.

I'm expecting that in fact $\hat{\theta} = \frac{\sum_i^n \log(X_i)}{n}$, and this will have gamma (sum of exponentials) distribution. To find the distribution, just find the MGF.