Understanding $\mathbf{\chi^2(k+1, \lambda_1)}$ and $\mathbf{\chi^2(n-k-1)}$

34 Views Asked by At

Show that $\mathbf{\frac{y^THy}{\sigma^2}\sim\chi^2(k+1, \lambda_1)} \text{ where } \mathbf{\lambda_1=\frac{\beta^TX^TX\beta}{2\sigma^2}} \text{ and that } \mathbf{\frac{y^T(I-H)y}{\sigma^2} \sim\chi^2(n-k-1)}$

I need help understanding the degrees of freedom for the chi-square distributions and how we get them. For example, I can simplify the LHS for both equation, i.e.

$$\mathbf{y^THy = y^TX(X^TX)^{-1}X^Ty = \beta^TX^TX\beta}$$ I believe there is a mistake in the question also as it should be $2\sigma^2$ on the LHS rather than $\sigma^2$ for the first equation. Essentially, what does the $k+1$ degrees of freedom suggest in this case?

Secondly, $$\mathbf{y^T(I-H)y = y^Ty-\beta^TX^TX\beta = SSE}$$

And why does this equate to $n-k-1$ degrees of freedom only?

1

There are 1 best solutions below

0
On BEST ANSWER

I figured out that because we have the $\mathbf{SSE}$ and that $s^2 = \frac{SSE}{n-k-1}$, we derive this by the following, if $E(y) = X\beta$ and $cov(y) = \sigma^2I$

We have that $$\mathbf{SSE = y^Ty - \beta^TX^Ty = y^T[I - X(X^TX)^{-1}X^T]y}$$

$$\begin{align}E(SSE) &= \mathbf{tr}\{[I - X(X^TX)^{-1}X^T]\sigma^2I\}+\beta^TX^T[I-X(X^TX)^{-1}X^T]X\beta \\ &={\sigma^2[n-tr(I_{k+1})] = \sigma^2(n-k-1)}\end{align}$$ Which is how we get $\chi^2(n-k-1)$ df for the second equation.

The $k+1$ comes about because $\beta^TX^TX\beta$ has a trace of $k+1$ across the diagonal.