Show inverting the LRT gives a confidence interval

485 Views Asked by At

Let $X_1, \ldots, X_n \sim \text{Exp}(\beta)$ be a random sample, where $\beta$ denotes the expectation. Derive a one sided $(1 - \alpha)$ confidence interval for $\beta$ by inverting the likelihood ratio test statistic.

Consider testing $H_0: \beta \leqslant \beta_0$ v.s. $H_1: \beta > \beta_0$. The likelihood ratio test statistic is \begin{equation} \lambda(\mathbf x) := \frac{ \sup_{\beta \leqslant \beta_0} L(\beta) }{ \sup_{\beta} L(\beta) } = \begin{cases} \left(\dfrac{\bar x}{\beta_0}\right)^n \exp\left( - n\bar x \left(\dfrac{1}{\beta_0} - \dfrac{1}{\bar x} \right) \right) & \text{if} \quad x > \beta_0, \\ 1 & \text{otherwise}, \end{cases} \end{equation}

where $\mathbf x := (x_1, \ldots, x_n)$ is an observed sample, $L(\beta)$ is the likelihood of $\beta$ given the sample and $\bar x := \frac{1}{n} \sum_i x_i$. We can define a $(1 - \alpha)$ confidence set $C_\alpha$ as \begin{equation} C_\alpha(\mathbf x) := \{ \beta_0 \ | \ \lambda(\mathbf x) \geqslant c(\beta_0)\} \quad \text{where} \quad \forall \beta_0: \mathbb P_{\beta_0} \{ \lambda(\mathbf X) < c(\beta_0) \} = \alpha. \end{equation}

By construction, $\mathbb P\{ \beta \in C_{\alpha}(\mathbf X) \} = 1 - \alpha$. It remains to show $C_\alpha$ is an interval. Taking the derivative of $\lambda(\mathbf x)$ with respect to $\beta_0$ shows it is monotone increasing with respect to $\beta_0$. If $c(\beta_0)$ were constant, we could have concluded $C_\alpha$ must therefore be of the form $[a, \infty)$ with $a$ nonnegative, and hence an interval.

Problem: How can we be sure $C_\alpha$ is an interval? I.e., can we prove that \begin{equation} \lambda(\mathbf x) - c(\beta_0) \end{equation}

increases monotonically with $\beta_0$?

Thanks in advance!

1

There are 1 best solutions below

0
On BEST ANSWER

Suppose $x=(x_1,x_2,\ldots,x_n)$ is the observed sample.

As you have found, the LRT statistic is

$$\lambda(x)=\begin{cases}\left(\frac{\bar x}{\beta_0}\right)^n e^{-n\left(\bar x/\beta_0-1\right)}&,\text{ if }\bar x>\beta_0 \\ 1 &,\text{ if }\bar x\le \beta_0\end{cases}$$

For $\bar x\le \beta_0$, we trivially accept $H_0$.

When $\overline x>\beta_0$, we reject $H_0$ if $\lambda(x)<c$ for some constant $c$.

Show that this critical region can be equivalently expressed as $\bar x>k$ for some $k=k(\beta_0,\alpha)$.

Since $X_i$ s are i.i.d exponential with mean $\beta$, we have $\frac2\beta X_i\stackrel{\text{i.i.d}}\sim \chi^2_2$.

Therefore a suitable test statistic is $$T=\frac{2}{\beta_0}\sum_{i=1}^n X_i=\frac{2n\overline X}{\beta_0}\stackrel{H_0}\sim\chi^2_{2n}$$

Now since $\overline x>k\implies \frac{2n\bar x}{\beta_0}>\frac{2nk}{\beta_0}$ and $P_{\beta_0}(T>\chi^2_{\alpha,2n})=\alpha$, we have $$\frac{2nk}{\beta_0}=\chi^2_{\alpha,2n}\implies k=\frac{\beta_0\chi^2_{\alpha,2n}}{2n}$$

Here $\chi^2_{\alpha,2n}$ is of course the $(1-\alpha)$th quantile of a $\chi^2_{2n}$ distribution.

So a level $\alpha$ LRT gives the critical region $\left\{x:\bar x>\frac{\beta_0\chi^2_{\alpha,2n}}{2n}\right\}$.

From the acceptance region of this test, we get

$$P_{\beta}\left[\overline X\le \frac{\beta\cdot\chi^2_{\alpha,2n}}{2n}\right]=1-\alpha\quad\,\forall\,\beta$$

This should give you the corresponding $100(1-\alpha)\%$ confidence interval for $\beta$.