Likelihood ratio test for Bernoulli distributions

1.1k Views Asked by At

Let's say that we have sample $x_1, ..., x_n \sim \textrm{Bernouli}(\theta)$ and consider hypothesis: $H_0: \theta_0 = \theta$ and $H_1: \theta_0 \neq \theta$. I want to find form of likelihood ratio in in this case.

Likelihood ratio connected with null hypothesis:

$$f_0(x) = \theta^{\sum x_i}(1 - \theta)^{n - \sum x_i}$$

Likelihood ratio connected with alternative hypothesis:

Let's fix $\theta_1 \neq \theta$, then:

$$f_1(x) = \theta_1^{\sum x_i}(1 - \theta_1)^{n - \sum x_i}$$

Likelihood ratio

Our likelihood ratio is then given as:

$$\frac{f_0(x)}{f_1(x)} = \frac{\theta^{\sum x_i}(1 - \theta)^{n - \sum x_i}}{\theta_1^{\sum x_i}(1 - \theta_1)^{n - \sum x_i}}$$

Am I correct with my way of thinking or I'm missing something?

1

There are 1 best solutions below

6
On BEST ANSWER

You need to specify a hypothesis of the form $\theta \in \Omega_0$ and the "alternative" (this is an unfortunate misnomer) is $\theta \in \Omega_1;$ contrary to what the name would suggest, you need to have $\Omega_0 \subset \Omega_1.$ In your case, $\Omega_1 = (0,1)$ and $\Omega_0 = \{\theta_0\}.$ Then, you need to consider the ratio of the two likelihoods maximased over the null and alternative hypotheses: $$ \Lambda = \dfrac{\sup\limits_{\theta \in \Omega_0} f(\theta)}{\sup\limits_{\theta \in \Omega_1} f(\theta)}. $$ In your case, $\Lambda = \dfrac{f(\theta_0)}{f(\theta_\mathrm{MLE})} = \dfrac{f(\theta_0)}{f(\bar X)}.$ Wilk's theorem asserts that $-2 \log \Lambda \approx \chi^2_{\mathrm{df}}$ if the sample size is large and where $\mathrm{df}$ is the difference of the dimension of $\Omega_1$ minus the dimension of $\Omega_0;$ in your case these dimensions are $1$ and $0,$ resp.

Thus, $-2 \log \Lambda \approx \chi^2_1.$

Additional remarks: note that $0 \leq \Lambda \leq 1$ so that $-2 \log \Lambda \in (0, \infty)$ and very large values of $-2 \log \Lambda$ are equivalent to very small values of $\Lambda.$ A very small value of $\Lambda$ signifies that the null hypothesis is much less likely than the "alternative" hypothesis; thus the alternative hypothesis ought to be preferred. (Note that the "null hypothesis" is, by definition, that hypothesis that you accept to be the "true" one in advance; only strong evidence against it should make you reconsider it for a different "alternative" hypothesis.) Wilk's theorem then asserts that one way to measure the degree of discrepancy between the null and alternative hypothesis is by the $\chi^2$-distribution. The amazin fact is that this distribution is universal (in the same sense that the normal distribution is universal in the $\mathsf{CLT}$).