Show regression result by matrix solving

42 Views Asked by At

I have following setup. 1 denotes an n dimensional column vector of ones and (1 X) denotes the $n \times (p+1)$ matrix with the first column being 1 and the remaining $n\times p$ block being X. We let

$$\tilde{X}=\frac{1}{n}X^T1$$ denote the p dimensional vector of mean values for the predictor and we let: $$\tilde{\Sigma}=\frac{1}{n}(x-1\tilde{X}^T)^T(X-1\tilde{X}^T)$$

I have to show that: $$n \tilde{\Sigma}=X^TX-n\tilde{X}\tilde{X}^T$$

I'm not totally sure how to show this. I think maybe I should use the rules: $(A+B)^T=A^T+B^T$ and that $(AB)^T=A^TB^T$. Can anyone help me? I maybe think that the n come from a multiply $1 \cdot 1^T$?

1

There are 1 best solutions below

0
On

First of all, $({\bf A}{\bf B})^{T} = {\bf B}^{T}{\bf A}^{T}$ rather than the one in the statement.

Since $\tilde{{\bf \Sigma}} = \frac{1}{n}({\bf X} - {\bf 1}{\tilde{\bf X}}^{T})^{T}({\bf X} - {\bf 1}{\tilde{\bf X}}^{T})$, we have \begin{align} n \tilde{{\bf \Sigma}} & = ({\bf X} - {\bf 1}{\tilde{\bf X}}^{T})^{T}({\bf X} - {\bf 1}{\tilde{\bf X}}^{T}) \\ & = {\bf X}^{T}{\bf X} - {\bf X}^{T}{\bf 1}{\tilde{\bf X}}^{T} - \tilde{\bf X}{\bf 1}^{T}{\bf X} + \tilde{\bf X}{\bf 1}^{T}{\bf 1}\tilde{\bf X}^{T} \\ & \triangleq term1 + term2 + term3 + term4, \end{align} where we define $term1 \triangleq {\bf X}^{T}{\bf X}$, $term2 \triangleq - {\bf X}^{T}{\bf 1}{\tilde{\bf X}}^{T}$, $term3 \triangleq - \tilde{\bf X}{\bf 1}^{T}{\bf X}$, and $term4 \triangleq \tilde{\bf X}{\bf 1}^{T}{\bf 1}\tilde{\bf X}^{T}$.

First, since $\tilde{\bf X} = \frac{1}{n}{\bf X}^{T}{\bf 1}$, we have ${\bf X}^{T}{\bf 1} = n\tilde{\bf X}$ and ${\bf 1}^{T}{\bf X} = ({\bf X}^{T}{\bf 1})^{T} = n\tilde{\bf X}^{T}$. Substituting these two equations into $term2$ and $term3$, we have $term2 = -n\tilde{\bf{X}}\tilde{\bf X}^{T}$ and $term3 = -n\tilde{\bf X}\tilde{\bf X}^{T}$.

Second, due to the fact that ${\bf 1}^{T}{\bf 1} = n$, we have $term4 = n\tilde{\bf X}\tilde{\bf X}^{T}$.

Finally, summing up four terms, we have \begin{align} n \tilde{{\bf \Sigma}} & = term1 + term2 + term3 + term4 \\ & = {\bf X}^{T}{\bf X} - n\tilde{\bf X}\tilde{\bf X}^{T} - n\tilde{\bf X}\tilde{\bf X}^{T} + n\tilde{\bf X}\tilde{\bf X}^{T} \\ & = {\bf X}^{T}{\bf X} - n\tilde{\bf X}\tilde{\bf X}^{T}. \end{align}