Determining whether $\hat{\beta}_1$ is unbiased

Question

The average annual temperature for 12 selected years in Berlin can be read from the following table:

Year	1890	1900	1911	1922	1930	1943
temperature	8.1	8.8	9.7	7.4	9.2	9.3

The second part of the table

Year	1966	1967	1983	1998	2000	2019
temperature	8.8	9.6	9.7	9.5	10.5	11.1

We will explain the temperature $Y_i$ in year $x_i$ using a simple linear regression model $$ Y_i = \beta_0 + \beta_1 x_i + \varepsilon_i, \; \; \; \; \; i = 1, ..., 12 $$ Where $\varepsilon_1, ... , \varepsilon_{12} \sim N(0, 0.5)$ are independent random variables and $\beta_0$, $\beta_1 \in \mathbb{R}$ are unknown parameters.

Is $\hat{\beta}_1$ unbiased? Find the $MSE(\hat{\beta}_1)$.

I have already found that $\hat{\beta}_0 = -23.9$ and $\hat{\beta}_1 = 0.017$. I have managed to solve it using R code, but i want to be able to do it by hand.

So far i know that $MSE(\hat{\Theta}) = E[(\hat{\Theta} - \theta)^2]$ and but i'm not sure i know what my $\hat{\Theta}$ and $\theta$ is here.

I havet tried to find information in my book about how i determine if $\hat{\beta}_1$ is unbiased, but haven't been able to find anything yet?

Answer 1

The MLE is indeed unbiased. In fact, more is true. We have $$ \hat \beta = (X^TX)^{-1}X^Ty$$ Thus $$E(\hat \beta \mid X) = (X^TX)^{-1}X^T E(y\mid X) = (X^TX)^{-1}X^TX\beta = \beta$$ Taking expectations on both sides shows $E(\hat \beta) = \beta$. Note the only assumptions we have used are $X^TX$ is invertible and $E(y\mid X) = X\beta$.

Answer 2

Note that i am referring to the book: Introduction to Probability Statistics and Random Processes by Hossein Pishro-Nik.

We have the following information from section 8.5.2: \begin{align*} \hat{\beta}_1 &= \frac{s_{xy}}{s_{xx}} \\ \bar{x} &= \frac{x_1 + x_2 + ... + x_n}{n} \\ \bar{y} &= \frac{y_1 + y_2 + ... + y_n}{n} \\ s_{xx} &= \sum_{i = 1}^{n} (x_i - \bar{x})^2 \\ s_{xy} &= \sum_{i = 1}^{n} (x_i - \bar{x})(y_i - \bar{y}) \end{align*} I can then use that to show that $\hat \beta_o$ is unbiased, by simplifying $E[\hat \beta_o]$ using information from section 8.5.2. \begin{align*} E[\hat \beta_1] &= \frac{s_{xy}}{s_{xx}} \\ &= \frac{\sum_{i = 1}^{n} (x_i - \bar{x})(y_i - \bar{y})}{\sum_{i = 1}^{n} (x_i - \bar{x})^2} \\ &= \frac{\sum_{i = 1}^{n} (x_i - \bar{x})\hat{y}_i}{\sum_{i = 1}^{n} (x_i - \bar{x})^2} \end{align*} % Above we have that $\hat{y}_i = (y_i - \bar{y})$ and since we also have $\hat{y}_i = \beta_0 + \beta_1 x_i$ (from the linear regression model), then we can substitute it in the equation: \begin{align*} &= \frac{\sum_{i = 1}^{n} (x_i - \bar{x})(\beta_0 + \beta_1 x_i)}{\sum_{i = 1}^{n} (x_i - \bar{x})^2} \\ &= \frac{ \beta_0 \sum_{i = 1}^{n} (x_i - \bar{x}) + \beta_1 \sum_{i = 1}^{n} (x_i - \bar{x}) x_i}{\sum_{i = 1}^{n} (x_i - \bar{x})^2} \end{align*} % Above I remove the parentheses, after which I multiply $x_i$ into $\sum_{i = 1}^{n} (x_i - \bar{x})$ below: \begin{align*} &= \frac{ \beta_0 \sum_{i = 1}^{n} (x_i - \bar{x}) + \beta_1 \sum_{i = 1}^{n} (x_i^2 - \bar{x}x_i)}{\sum_{i = 1}^{n} (x_i - \bar{x})^2} \\ &= \beta_1 \sum_{i = 1}^{n} (x_i^2 - \bar{x}x_i) \end{align*} Above we realized that the term $\beta_0 \sum_{i = 1}^{n} (x_i - \bar{x}) = 0$ because the term $\sum_{i = 1}^{n} (x_i - \bar{x})$ is just the sum of a data set's deviations from their mean value and it is known that the sum of this gives zero. The same applies to the term $\sum_{i = 1}^{n} (x_i - \bar{x})^2$ \ % Now I use the summation rule to get the following: \begin{align*} &= \beta_1 \bigg(\sum_{i = 1}^{n} (x_i^2) - \sum_{i = 1}^{n} (\bar{x}x_i)\bigg) \\ &= \beta_1 \bigg(\sum_{i = 1}^{n} (x_i^2) - n\bar{x}^2\bigg) \end{align*} Above we remembered that $\sum_{i=1}^{n} x_i = n\bar{x}$, so we replaced the value $\sum_{i=1}^{n} x_i\bar{x}$ with $n\bar{x}\bar{x} = n\bar{x}^2$. Now I realize that $\sum_{i=1}^n x_i^2 = (n\bar{x}^2)$ and thus get the following result: \begin{align*} &= \beta_1 \bigg(n\bar{x}^2 - n\bar{x}^2\bigg) \\ &= \beta_1 \end{align*} Therefore i get: $$ E[\hat \beta_1] = \beta_1 $$ This means that $\hat{\beta}_1$ is unbiased.