Correlation between Least Square Parameters obtained in Linear Regression

803 Views Asked by At

I am trying to solve Q 22 below:

Question

Upon minimizing the squared errors I get the following values of the parameters:

$$\hat{\alpha} = \overline{y}$$

$$\hat{\beta} = \frac{\sum x_i y_i - \overline{x}\sum y_i}{\sum (x_i - \overline{x})^2}$$

How should one conclude the value of correlation coeff from here?

1

There are 1 best solutions below

2
On BEST ANSWER

$\newcommand{\bhat}{\widehat{b}}\newcommand{\ahat}{\widehat{a}}\newcommand{\Sxy}{S_{xy}}\newcommand{\Sxx}{S_{xx}}\newcommand{\Var}{\operatorname{Var}}\newcommand{\Cov}{\operatorname{Cov}}$Hints: We use some relatively standard notations. Denote $S_{xy} = \sum\limits_{i=1}^n (x_i - \bar{x})y_i$ and $S_{xx} = \sum\limits_{i=1}^n (x_i-\bar{x})^2$. Then $\bhat = \frac{\Sxy}{\Sxx}$ and $\ahat = \frac{1}{n}\sum\limits_{j=1}^{n}y_j$. Hence $$\begin{align} \Cov(\ahat, \bhat)&=\Cov(\bar{y} , \bhat)\\ &= \Cov\left(\frac{1}{n}\sum\limits_{j=1}^n y_j, \frac{\sum\limits_{i=1}^{n}(x_i-\bar{x})y_i}{\Sxx}\right) \\ &= \frac{1}{n \Sxx}\sum\limits_{j=1}^{n}\sum\limits_{i=1}^{n}(x_i-\bar{x})\Cov\left(y_j , y_i\right).\quad(\text{covariance is bilinear}) \end{align}$$

Now, split the double sum into cases: for $i\ne j$, the covariance between $y_i$ and $y_j$ is $0$. And for $i=j$, $\Cov(y_i, y_i)=\Var(y_i)=1$. Therefore, noting that $\sum\limits_{i=1}^{n}(x_i-\bar{x})$ has a simple value (what is it?), what is the value of the above double sum, and hence of the correlation between $\ahat$ and $\bhat$?