Linear regression model with a known parameter

Question

Question:

Consider the following linear regression with one parameter (intercept $\beta_{0}$ known).

$ y_{i} = \beta_{0}^{*} + \beta_{1}x_{i} + \epsilon_{i} $ for i = 1,....n

a) Compute the LSE , MLE , mean and varience for $\beta_{1}$.

I understand how to derive the following when both the slope and intercept is unknown. However I'm unsure what the effect of knowing what the intercept will have on the new computations.

Attempt: We minimsise the LSE by setting

$\partial_{\beta_{1}} S(\beta_{1}) = - \sum_{i=1}^{n} 2x_{i}(y_{i}-\beta_{0}^{*}-\beta_{1}x_{i}) = 0 $

After long computations we get

$\beta_{1} = \frac{\sum_{i=1}^{n} x_{i} (y_{i} - \beta_{0}^{*}) }{\sum_{i=1}^{n}(x_{i}^2)}$

The computation of the least square estimator doesn't seem to care wether $b_{0}$ is known or unknown so would be get the same LSE,MLE,mean and variance as with the case $b_{0}$ is unknown

Answer 1

Hint

Do exactly what is done with the two parameters $\beta_0,\beta_1$ but with only $\beta_1$ as a parameter. For example, it means that the partial derivative $\frac{\partial}{\partial \beta_1}$ is in fact a usual derivative depending of $x_i, y_i$ and $\beta_0$ which is supposed to be known.

Answer 2

Since $\beta _{\,0} ^ * $ is determined, we have better and consider $$ z_{\,i} = y_{\,i} - \beta _{\,0} ^ * $$

Then the error squared is $$ E^{\,2} (\beta _{\,1} ) = \sum\limits_i {\left( {z_{\,i} - \beta _{\,1} x_{\,i} } \right)^{\,2} } $$ and it gets minimized when $$ \eqalign{ & {d \over {d\beta _{\,1} }}E^{\,2} (\beta _{\,1} ) = 0\quad \to \quad \sum\limits_i {x_{\,i} \left( {z_{\,i} - \beta _{\,1} x_{\,i} } \right)} = 0\quad \to \cr & \to \quad \beta _{\,1} = {{\sum\limits_i {x_{\,i} z_{\,i} } } \over {\sum\limits_i {x_{\,i} ^2 } }} \cr} $$

This is similar to what you got, except that the denominator involves the raw second moment of the $x_i$.

Thereafter, to analize the reliability (possible error) on the estimated parameter $\beta _{\,1}$ we proceed as follows.

Suppose that the true line is $$ \hat z = \hat \beta _{\,1} x $$ and, since the $x_i$ are considered "precise", the $z_i$ be subject instead to deviation wrt to the true value $ \hat z_{\,i}$ $$ z_{\,i} = \hat z_{\,i} + \varepsilon _{\,i} = \hat \beta _{\,1} x_{\,i} + \varepsilon _{\,i} $$ then $$ \eqalign{ & \beta _{\,1} = {1 \over {\sum\limits_i {x_{\,i} ^2 } }}\sum\limits_i {x_{\,i} z_{\,i} } = {1 \over {\sum\limits_i {x_{\,i} ^2 } }}\sum\limits_i {x_{\,i} \left( {\hat \beta _{\,1} x_{\,i} + \varepsilon _{\,i} } \right)} = \cr & = \hat \beta _{\,1} + \sum\limits_i {{{x_{\,i} } \over {\sum\limits_i {x_{\,i} ^2 } }}\varepsilon _{\,i} } \cr} $$ so the error of $beta_1$ is a linear combination of the errors in $z$.

Therefore, assuming that the intercept $\beta _{\,0} ^ * $ was precise, we can assume that the $\varepsilon _{\,i} $ be random variables with null mean. And if they are i.i.d., the variance of $\beta_1-\hat \beta _{\,1} $ follows easily.

In particular, $$ z_{\,i} = \hat z_{\,i} + \varepsilon _{\,i} = \hat \beta _{\,1} x_{\,i} + \varepsilon _{\,i} \quad \to \quad E\left[ {z_{\,i} } \right] = \hat \beta _{\,1} E\left[ {x_{\,i} } \right] + E\left[ {\varepsilon _{\,i} } \right] $$ and if the $\varepsilon _{\,i} $ have null mean then, introducing the sample averages indicated with the over bar, we shall have $$ \bar z_{\,i} \approx \hat \beta _{\,1} \bar x_{\,i} \approx \beta _{\,1} \bar x_{\,i} $$ which means that the line shall pass (almost) through the barycenter of the cloud of points.
That is the case in fact, when you compute the regression line with two parameters. Practically you do that by shifting the origin of the coordinates at the barycenter and then computing the slope as above, now using the central moments.

If instead the $ \varepsilon _{\,i} $ actually have a mean non null $$ \hat z = \hat \beta _{\,1} x + \hat \mu \quad \quad z_{\,i} = \hat z_{\,i} + \varepsilon _{\,i} = \hat \beta _{\,1} x_{\,i} + \hat \mu + \varepsilon _{\,i} $$ then in the calculation above we are introducing an error $$ \eqalign{ & \beta _{\,1} = {1 \over {\sum\limits_i {x_{\,i} ^2 } }}\sum\limits_i {x_{\,i} z_{\,i} } = {1 \over {\sum\limits_i {x_{\,i} ^2 } }}\sum\limits_i {x_{\,i} \left( {\hat \beta _{\,1} x_{\,i} + \varepsilon _{\,i} + \hat \mu } \right)} = \cr & = \left( {\hat \beta _{\,1} + \hat \mu {{\sum\limits_i {x_{\,i} } } \over {\sum\limits_i {x_{\,i} ^2 } }}} \right) + \sum\limits_i {{{x_{\,i} } \over {\sum\limits_i {x_{\,i} ^2 } }}\varepsilon _{\,i} } \cr} $$