Confidence interval of the parameter of $\exp$ and normal distribution from MLE?

Question

I have a sample $X_1,X_2,\ldots,X_n$

1. If the sample is from exponential distribution, I want to use MLE to estimate the parameter $\beta$. I know the result that $$\hat{\beta}=\frac{X_1+X_2+\ldots+X_n}{n}$$But how can I calculate the $95\%$ confidence interval of $\beta$?

2. If the sample is from normal distribution, I want to use MLE to estimate the parameter $\mu$ and $\sigma$. I also know the result that $$\hat{\mu}=\frac{X_1+X_2+\ldots+X_n}{n}, \quad \hat{\sigma}=\left[\frac{n-1}{n}\cdot S^2(n)\right]^{1/2}$$ But how can I calculate the $95\%$ confidence interval of $\mu$ and $\sigma$?

   1. Step1 to calculate confidence interval, $$\hat{θ}\pm z_{1-\frac\alpha2}\sqrt{\frac{\delta(\hat{θ})}{n}}$$
   2. Step2 to calculate confidence interval $$\delta(\hat{θ})=-n\left(E\left[\frac{d^2}{dθ^2}\ln \mathcal L(θ)\right]\right)^{-1}$$

I know the equation, but I am still confuzed how to calculate $$\left(E\left[\frac{d^2}{dθ^2}\ln \mathcal L(θ)\right]\right)$$

I want to know the equation in the red frame of this picture.

Answer 1

The term you cannot calculate is (essentially) the Fischer information:

$$I(θ)=-E\left[\frac{d^2}{dθ^2}\ln \mathcal L(θ)\right]$$

Then $δ(θ)=n(I(θ))^{-1}$. So, to calculate $I(θ)$ for the exponential:

$$\mathcal L(θ)=\prod\limits_{j=1}^nθ e^{-θx_j}=θ^n e^{-θ\sum_{j=1}^{n}x_j}$$ and therefore $$\ln \mathcal L(θ) = \ln{\left(θ^n e^{-θ\sum_{j=1}^{n}x_j}\right)} = n\ln{(θ)}-θ\sum_{j=1}^{n}x_j$$ which you can differentiate twice with respect to $θ$: \begin{align}\frac{d}{dθ} \ln (\mathcal L(θ)) &= \frac{n}{θ}-\sum_{j=1}^n x_j \\ \frac{d^2}{dθ^2} \ln (\mathcal L(θ)) &= -\frac{n}{θ^2}\end{align} So, $$I(θ)=-E\left[\frac{d^2}{dθ^2}\ln \mathcal L(θ)\right]=\frac{n}{θ^2}$$ and therefore $δ(θ)=n(n/θ^2)^{-1}=θ^2$. To make calculations simpler use that

If $X_j$ are i.i.d. for $j=1,2,\ldots,n$ you can take the $I(θ)$ for a single observation $X_j$ and obtain the Fisher information for $X$ with $nI(θ)$.

---

For the normal distribution $N(\mu,θ^2)$ you should find: $I(θ)=n/(2θ^2)$ (single parameter) and if you estimate both parameters, i.e. $Ν(μ,σ^2)$ with $θ=[μ,σ^2]^Τ$ then $I(θ)$ is a matrix: $$I(θ)=\begin{bmatrix}\frac{1}{2σ^2}&0 \\ 0& \frac{1}{2σ^4}\end{bmatrix}$$