Likelihood ratio criterion for testing equality of means in 2 normal populations

331 Views Asked by At

Let assume we have the following $$ X_{11}, X_{12},...,X_{1n_{1}} \sim N_{p}(\mathbf{\mu_{1}},\mathbf{\Sigma_{1}}) \\ X_{21}, X_{22},...,X_{2n_{2}} \sim N_{p}(\mathbf{\mu_{2}},\mathbf{\Sigma_{2}}) $$ We are testing the hypothesis that $$ H_{0}: \mathbf{\mu}_{1} = \mathbf{\mu}_{2} $$
vs $$ H_{1}: \mathbf{\mu}_{1} \neq \mathbf{\mu}_{2} $$ under the assussmtion that $$\mathbf{\Sigma_{1}}\neq \mathbf{\Sigma_{1}}$$

So under the null hypothesis $H_{0}$: $\mathbf{\mu_{1}} = \mathbf{\mu_{2}}=\mathbf{\mu}$

The Likelihood ratio test is given by: $$\Lambda(x)=\frac{\max_{\mathbf{\Sigma_{1},\Sigma_{2}}} \mathcal{L}( \mathbf{\mu}, \mathbf{\Sigma_{1}}, \mathbf{\Sigma_{2}})}{\max_{\mathbf{\mu,\Sigma_{1},\Sigma_{2}}} \mathcal{L}( \mathbf{\mu}, \mathbf{\Sigma_{1}}, \mathbf{\Sigma_{2}})}$$

\begin{align} \mathcal{L}_{0}(\mathbf{\mu}, \mathbf{\Sigma_{1}}, \mathbf{\Sigma_{2}})= (2 \pi)^{\frac{-p \, n_{1}}{2}} \, \exp\left\{\frac{-1}{2} \sum_{i=1}^{n_{1}} (X_{i1} - \mathbf{\mu})^{T} \mathbf{\Sigma_{1}}^{-1} (X_{1i}- \mathbf{\mu}) \right\} \times (2 \pi)^{\frac{-p \, n_{2}}{2}} \, \exp\left\{\frac{-1}{2} \sum_{j=1}^{n_{2}} (X_{2j} - \mathbf{\mu})^{T} \mathbf{\Sigma_{2}}^{-1} (X_{2j}- \mathbf{\mu}) \right\} \end{align}

I am not really sure where to go from here and how is my denominator will look likes. Thank you in advance.

1

There are 1 best solutions below

0
On

Given samples $x_1$ and $x_2$, where $x_k\equiv \{x_{ki}\}_{i=1}^{n_k}$, let $$ \bar{x}_k:=\frac{1}{n_k}\sum_{i=1}^{n_k}x_{ki} \quad\text{and}\quad S_k:=\frac{1}{n_k}\sum_{i=1}^{n_k}(x_{ki}-\bar{x}_k)(x_{ki}-\bar{x}_k)^{\top}. $$ The log-likelihood is given by $$ \ln\mathcal{L}(\mu_1,\mu_2,\Sigma_1,\Sigma_2\mid x_1,x_2)=\ln\mathcal{L}_1(\mu_1,\Sigma_1\mid x_1)+\ln\mathcal{L}_2(\mu_2,\Sigma_2\mid x_2), $$ where $$ \bbox[LightGreen,5px] { \ln\mathcal{L}_k(\mu,\Sigma\mid x_k)=-\frac{n_kp}{2}\ln(2\pi)+\frac{n_k}{2}\ln|\Sigma^{-1}|-\frac{1}{2}Q_k(\mu,\Sigma) } $$ with \begin{align} Q_k(\mu,\Sigma)&:=\sum_{i=1}^{n_k}(x_{ki}-\mu)^{\top}\Sigma^{-1}(x_{ki}-\mu)=\sum_{i=1}^{n_k}\operatorname{tr}\left((x_{ki}-\mu)^{\top}\Sigma^{-1}(x_{ki}-\mu)\right) \\ &=\sum_{i=1}^{n_k}\operatorname{tr}\left(\Sigma^{-1}(x_{ki}-\mu)(x_{ki}-\mu)^{\top}\right)=\operatorname{tr}\left(\Sigma^{-1}\sum_{i=1}^{n_k}(x_{ki}-\mu)(x_{ki}-\mu)^{\top}\right)\\ &=n_k\operatorname{tr}\left(\Sigma^{-1}(S_k+(\bar{x}_k-\mu)(\bar{x}_k-\mu)^{\top}\right) \\ &=n_k\operatorname{tr}(\Sigma^{-1}S_k)+n_k(\bar{x}_k-\mu)^{\top}\Sigma^{-1}(\bar{x}_k-\mu). \end{align}


Since $\Sigma^{-1}$ is positive-definite, the second term of $Q_k(\mu,\Sigma)$ is positive for any $\mu\ne \bar{x}_k$. Therefore, $\ln\mathcal{L}_k(\mu,\Sigma\mid x_k)$ is maximized when $\mu=\bar{x}_k$, i.e. the MLE of $\mu_k$ is $$ \bbox[cornsilk,5px] { \hat{\mu}_k=\bar{x}_k. } $$ Differentiating the concentrated log-likelihood w.r.t. $\Sigma^{-1}$, one gets $$ \frac{\partial\ln\mathcal{L}_k(\hat{\mu}_k,\Sigma\mid x_k)}{\partial \Sigma^{-1}}=\frac{n_k}{2}\frac{1}{|\Sigma^{-1}|}\frac{\partial |\Sigma^{-1}|}{\partial \Sigma^{-1}}-\frac{n_k}{2}\frac{\partial\operatorname{tr}(\Sigma^{-1}S_k)}{\partial \Sigma^{-1}}=\frac{n_k}{2}\Sigma-\frac{n_k}{2}S_k. $$ Thus, the MLE of $\Sigma_k$ is $$ \bbox[cornsilk,5px] { \hat{\Sigma}_k=S_k, } $$ and so $Q_k(\hat{\mu}_k,\hat{\Sigma}_k)=n_k\operatorname{tr}(I_p)=n_kp$.


Under $H_0$ when $\mu_1=\mu_2=\mu$, the MLE of $\Sigma_k$ is $$ \bbox[cornsilk,5px] { \hat{\Sigma}_{0k}=S_{0k}:=\frac{1}{n_k}\sum_{i=1}^{n_k}(x_{ki}-\mu)(x_{ki}-\mu)^{\top} } $$ because $$ \frac{\partial\ln\mathcal{L}_k(\mu,\Sigma\mid x_k)}{\partial \Sigma^{-1}}=\frac{n_k}{2}\frac{1}{|\Sigma^{-1}|}\frac{\partial |\Sigma^{-1}|}{\partial \Sigma^{-1}}-\frac{1}{2}\frac{\partial Q_k(\mu,\Sigma)}{\partial \Sigma^{-1}}=\frac{n_k}{2}\Sigma-\frac{n_k}{2}S_{0k}. $$

Similarly to the previous case, $Q_k(\mu,\hat{\Sigma}_{0k})=n_k\operatorname{tr}(\hat{\Sigma}_{0k}^{-1}S_{0k})=n_kp$.


Combining these results, one obtains $$ \bbox[lightsteelblue,5px] { \Lambda(x_1,x_2)=\frac{\mathcal{L}(\mu,\mu,\hat{\Sigma}_{01},\hat{\Sigma}_{02}\mid x_1,x_2)}{\mathcal{L}(\hat{\mu}_1,\hat{\mu}_2,\hat{\Sigma}_1,\hat{\Sigma}_2\mid x_1,x_2)}=\left(\frac{|\hat{\Sigma}_1|}{|\hat{\Sigma}_{01}|}\right)^{\frac{n_1}{2}}\left(\frac{|\hat{\Sigma}_2|}{|\hat{\Sigma}_{02}|}\right)^{\frac{n_2}{2}}. } $$