Distribution of e, least squares residuals

Question

I have the model $y=X{\beta}+{\epsilon}$ and $E({\epsilon})=0$ and $Var({\epsilon})={\sigma}^2I_n$

The vector y can be written: $y=X\hat{\beta}+{e}$

If ${\epsilon}$~$N(0, {\sigma}^2I_n)$ how is $e^Te$ distributed?

Answer 1

$\frac{e^Te}{\sigma^2} \sim \chi^2_{n-k}$ with $X \in \mathbb{R}^{n \times k}$ of full rank.

Let $\Pi_X$ denote the orthogonal projection on the column-space of X. Then $X\hat{\beta} = \Pi_X $ and $ e = y - X\hat{\beta} = (I_n - \Pi_x)y$

As a projection matrix, $(I_n - \Pi_x)$ is symmetric, idempotent, and can be diagonalized in the following way:

$(I_n - \Pi_x) = P D_{n-k} P^T$ with an orthogonal matrix P and

$ D_{n-k} = \begin{bmatrix} 1 & 0 & 0 & \cdots & 0 \\ 0 & 1 & 0 & \cdots & 0 \\ \vdots & \\ 0 & 0 & 0 & \cdots & 0 \end{bmatrix} $ having $(n-k)$ ones on the main diagonal and zeros elsewhere.

Define $ Z:=\frac{P^T \varepsilon}{\sigma} \sim N(0, P^T I_n P) ) = N(0, I_n)$.

$\frac{e^Te}{\sigma^2} = \frac{y^T}{\sigma}(I_n - \Pi_x)(I_n - \Pi_x)^T \frac{y}{\sigma} = \frac{\varepsilon^T}{\sigma}(I_n - \Pi_x) \frac{\varepsilon}{\sigma} = \frac{\varepsilon^T}{\sigma} P D_{n-k} P^T\frac{\varepsilon}{\sigma} \stackrel{d}{=} \sum_{i=1}^{n-k} Z_i^2 \sim \chi^2_{n-k} $

Hope it helps

skeletor