MLE of Multivariate Normal Distribution

714 Views Asked by At

Let $x_1,...,x_n$ be a random sample from a multivariate normal distribution with mean $\mu$ and covariance matrix $\Sigma$. I want to show that $$ \hat{\mu}=\bar{x}\quad\text{ and }\quad\hat{\Sigma}=\frac{1}{n}(x_i-\mu)'(x_i-\mu) $$ are the maximum likelihood estimators, i.e. I want to show that \begin{align} \prod_{i=1}^n\frac{1}{(2\pi)^{\frac{p}{2}}\det(\Sigma)^{\frac{1}{2}}}exp\bigg(-\frac{1}{2}(x_i-\mu)'\Sigma^{-1}(x_i-\mu)\bigg) =\frac{1}{(2\pi)^{\frac{np}{2}}\det(\Sigma)^{\frac{n}{2}}}exp\bigg(-\frac{1}{2}\sum_{i=1}^n(x_i-\mu)'\Sigma^{-1}(x_i-\mu)\bigg) \end{align} is maximized by $\hat{\mu}$ and $\hat{\Sigma}$ defined above. We first check for the minimizer of $\mu$. For this only the second part of the term is interesting. I do the trace trick: \begin{align} \sum_{i=1}^n\bigg(x_i-\mu)'\Sigma^{-1}(x_i-\mu) \bigg) &= \sum_{i=1}^n\bigg( tr\Big((x_i-\mu)'\Sigma^{-1}(x_i-\mu)\Big)\bigg)\\ &= \sum_{i=1}^n\bigg( tr\Big(\Sigma^{-1}(x_i-\mu)(x_i-\mu)'\Big)\bigg)\\ &= \sum_{i=1}^n\bigg( tr\Big(\Sigma^{-1}(x_i-\bar{x}+\bar{x}-\mu)(x_i-\bar{x}+\bar{x}-\mu)'\Big)\bigg)\\ &= tr\bigg(\Sigma^{-1}\sum_{i=1}^n\Big((x_i-\bar{x}+\bar{x}-\mu)(x_i-\bar{x}+\bar{x}-\mu)'\Big)\bigg)\\ &= tr\bigg(\Sigma^{-1}\sum_{i=1}^n\Big((x_i-\bar{x})(x_i-\bar{x})'\Big)+n(\bar{x}-\mu)(\bar{x}-\mu)'\bigg),\\ \end{align} where I used in the last step that the cross terms cancel each other out. It es now easy to see that the maximizer of $\mu$ is given by $\hat{\mu}=\bar{x}$. But I could not prove that $\Sigma$ is maximized by $\hat{\Sigma}=\frac{1}{n}(x_i-\mu)'(x_i-\mu)$. I would be glad for any help.