Variance of standardized linear regression coefficient

Question

Suppose we have standardized inputs (mean $0$, variance $1$). So that $y=x_1\beta_1+x_2\beta_2+\varepsilon$, (we have no $\beta_0$ due to standardization). With $\varepsilon \sim N(0,\sigma^{2})$. Also note that we only have 2 variables.

We wish to show that $Var(\beta_1)=\frac{\sigma^{2}}{1-r_{12}^{2}}$, where $r_{12}$ is the sample correlation between $x_1,x_2$.

So I have arrived at $Var(\hat{\beta})=Var((X^{T}X)^{-1}X^{T}Y)=Var((X^{T}X)^{-1}X^{T}(X\beta+\varepsilon))=Var((X^{T}X)^{-1}X^{T}\sigma^2I) = \sigma^2(X^TX)^{-1}$

And I'm not exactly sure where to continue, any help would be appreciated.

Answer 1

I am not very comfortable with Mathjax. I provided the solution in the below image, but I believe that there is some missing term in the variance that you mentioned in your question. If we assume there are n terms available for each dependent and independent variable then

Answer 2

It is a result in regression analysis.

We denote the design matrix $X = (x_1, x_2)$, where $x_1, x_2 \in \mathbb{R}^n$ supposing we have $n$ sample units (here may be some abuse of notaions since there are $x_1$ and $x_2$ in your problem).

The covariance matrix of estimator of $\beta$ which is already derived in your post is $$ Var(\hat\beta) = (X^TX)^{-1}\sigma^2 $$

We apply the rule of multipication of block matrix, $$ \begin{align} (X^T X)^{-1} &= \left( \begin{pmatrix} x_1^T \\ x_2^T \end{pmatrix} \begin{pmatrix} x_1 x_2 \end{pmatrix} \right)^{-1} \\ &= \left( \begin{pmatrix}x_1^Tx_1 & x_1^Tx_2 \\ x_2^T x_1 & x_2^T x_2 \end{pmatrix} \right)^{-1} \\ &= \begin{pmatrix}\frac{1}{x_1^Tx_1(1 - \frac{(x_1^Tx_2)^2}{x_1^Tx_1 x_2^T x_2})} & * \\ * & * \end{pmatrix} \end{align} $$ where the last equation is derived by the inverse of block matrix equation(2.3).

The $x_1$ and $x_2$ has been standardized then $x_1^T x_1 = -n-1, x_2^Tx_2 = n-1$ and the correlation $r_{12} = cor(x_1, x_2) = \frac{x_1^T x_2}{\sqrt{x_1^Tx_1 x_2^Tx_2}}$.

Hence, $Var(\hat\beta_1) = \frac{\sigma^2}{(n-1)(1 - r_{12}^2)}$.

Btw, you got stuck just because of some trick of algebra. The Matrix Cookbook may be a good reference.