Why this matrix is positive semi-definite?The difference between RLS estimator and OLS estimator with respect to their variance.

158 Views Asked by At

It seems not obvious to me and I want to know why the matrix C is positive semi-definited ?

See the source : http://web.vu.lt/mif/a.buteikis/wp-content/uploads/PE_Book/4-4-Multiple-RLS.html

Let L be a 'r x k' constants matrix and rank(L)= r, Let X be a 't x k' constants matrix and rank(X)= k;

\begin{aligned} \mathbb{V}{\rm ar} \left(\widehat{\boldsymbol{\beta}}^{(OLS)} \right) - \mathbb{V}{\rm ar} \left(\widehat{\boldsymbol{\beta}}^{(RLS)} \right) &= \sigma^2 \left( \mathbf{X}^\top \mathbf{X}\right)^{-1} \mathbf{L}^\top \left( \mathbf{L}\left( \mathbf{X}^\top \mathbf{X} \right)^{-1} \mathbf{L}^\top \right)^{-1} \mathbf{L} \left( \mathbf{X}^\top \mathbf{X}\right)^{-1} = \sigma^2 \mathbf{C} \end{aligned}

Here the matrix C

1

There are 1 best solutions below

2
On

$X^\top X$ is positive definite; so is its inverse.

Since $(X^\top X)^{-1}$ is positive definite, we have $v^\top L (X^\top X)^{-1} L^\top v (L^\top v)^\top (X^\top X)^{-1} (L^\top v) \ge 0$ for any $v$. So $L(X^\top X)^{-1} L^\top$ is positive semi-definite.

Based on your expression it seems $L(X^\top X)^{-1} L^\top$ is also invertible. (I don't think this fact can be deduced from the information you provided.) If so, then this inverse is positive definite.

From there, you can conclude that $$v^\top (X^\top X)^{-1} L^\top (L(X^\top X)^{-1} L^\top)^{-1} L(X^\top X)^{-1} v = w^\top (L(X^\top X)^{-1} L^\top)^{-1} w \ge 0$$ for any $v$, where $w := L(X^\top X)^{-1} v$, so your matrix is positive semi-definite.