If I have this linear regression equation:
$$y=X\beta+\epsilon $$
($x$ and $\beta$ are vectors)
The likelihood function can be written as
$$L= \prod_{n=1}^N N(y_n ;x_n ,\beta ,\sigma^2)=(2\pi \sigma^2)^{-\frac {N}{2}}\exp\left\{ \frac{-1}{2\sigma^2} (y- X\beta)'(y-X\beta)\right\}$$
However, what changes if the $\epsilon$ term is independent but non-identically normally distributed? I know it would mean that the mean is the same but variance of the noise changes.
How do I represent it in the likelihood function to proceed with MLE of $\beta$?
- Do I simply take the sum of all variances in the term above? Like $\frac{\sum\sigma^2_n}{N}$?
The generalization is as follows:
$$ L = \frac{1}{\sqrt{(2\pi)^n \left|\Sigma \right|}} \exp \left\{ -\frac{1}{2}(y-X\beta)^T \Sigma^{-1}(y-X\beta)\right\} $$
where $\Sigma$ is the covariance matrix of the multivariate Gaussian distribution, and $|\cdot|$ denotes its determinant. If the noises are independent, then it will be a diagonal matrix with noise variances on the diagonal.