Covariance matrix of least square estimator with measurements having different variances

56 Views Asked by At

If I have a linear least square fit, where the $y$ values all have different variances, than the estimator for the parameter is written as follow: $$ \vec{\theta} = (A^T V^{-1}A)^{-1}A^TV^{-1}\vec{y} $$

Apparently (because a question on a worksheet is asking it), using standard error propagation, one can show, that the variance is $$ V(\vec{\theta}) = (A^TV^{-1}A)^{-1} $$ I cant work this out myself, and so far, all the proofs I have seen for the Variance of the linear least square fits were, when the $y_i$ all having the same Variance, but I need this more general form.

1

There are 1 best solutions below

0
On BEST ANSWER

Note that $var(Y) = V$ and denote $B = (A'V^{-1}A)^{-1}A'V^{-1}$, hence \begin{align} var(\hat{\theta}) &= var(By)\\ &= Bvar(y) B^T\\ & = BVB^T\\ &= (A'V^{-1}A)^{-1}A'V^{-1} V V^{-1}A(A'V^{-1}A)^{-1}\\ &= (A'V^{-1}A)^{-1}(A'V^{-1}A)(A'V^{-1}A)^{-1}\\ &= (A'V^{-1}A)^{-1}. \end{align}