Asymptotic variance of estimators of regression coefficients

656 Views Asked by At

When doing multiple linear regression,

$\boldsymbol{Y} = X\boldsymbol{\beta}+\boldsymbol{\epsilon}$

where

$\boldsymbol{\epsilon} \sim N(0, \Sigma)$

The best estimates of the coefficients can be found using the generalised least squares formula:

$\boldsymbol{\hat{\beta}} = (X'\Sigma^{-1}X)^{-1}X'\Sigma^{-1}\boldsymbol{Y}$.

From this we can find:

$Var(\boldsymbol{\hat{\beta}}) = (X'\Sigma^{-1}X)^{-1}$

Alternatively, we know that the asymptotic distribution of the estimators is given by

$\boldsymbol{\hat{\beta}} \sim N(\boldsymbol{\beta}, I^{-1})$

where I is the information matrix, and hence

$Var(\boldsymbol{\hat{\beta}}) = I^{-1}$

I have been working through a problem and have been calculating the best estimates of the coefficients using two methods, either by maximising the log-likelihood numerically, or by using generalised least squares (GLS). In both cases I get the same estimates for the parameters, however the covariance matrices are very different, ie I would expect

$(X'\Sigma^{-1}X)^{-1} \approx I^{-1}$

however I am finding that the variances are much smaller when using the Information matrix. This is obviously causing issues when doing hypothesis tests etc. I am simulating my own data and so know what to expect when doing these tests, and it works nicely when using GLS, but the lower variance from the Information matrix is giving me inconsistent results.

Any ideas why this is happening?