I'm trying to understand simple linear regression. Here is a problem I'm working on, and I'm trying to understand the answers conceptually.
I want to find the least square estimators $b_1$ and $b_0$ for the following:
(a) The "no slope" model $Y_i = \beta_0 + \epsilon_i$, and
(b) The "no intercept" model $Y_i = \beta_1x_i + \epsilon_i$.
To do this, I know that I will need to minimize the function $Q = \sum_{i=1}^{n} \epsilon_i^{2}$. Thus, I have
(a) $Q = \sum_{i=1}^{n} \epsilon_i^{2} = (Y_i-\beta_0)^2$. Now minimizing this yields,
$0=\frac{\partial Q}{\partial \beta} = \sum_{i=1}^{n} -2(Y_i-b_0) \Rightarrow 0=\sum_{i=1}^{n}Y_i-nb_0 \Rightarrow b_0 = \overline{Y}$.
(b) similarly for this part, I get that $b_1 = \frac{\sum_{i=1}^{n}y_ix_i}{\sum_{i=1}^{n} x_i^{2}}$.
I understand that part (a) makes intuitive sense -- if there is no slope, then the regression line is a horizontal line with value $\beta_0$. So, even though $\beta_0$ is an unknown parameter, we can still reason that the best estimate would be whatever its value is. I'm looking for a way to understand the answer for part (b) in the same way (it seems less intuitive to me). Thanks for any explanation.
One answer to your question about (b) is that this is the way the math works out - nothing more, nothing less.
Since this is probably unsatisfactory, I have come up with this:
The slope for each point is $\dfrac{y_i}{x_i}=s_i$. Therefore, if the result is of the form $\dfrac{\sum u_i}{\sum v_i}$, then we "should" have $u_i = v_is_i$, so the slopes weight the data. If $s_i = s$ for all $i$, then we want $s =\dfrac{\sum u_i}{\sum v_i} =\dfrac{\sum v_is_i}{\sum v_i} =s $, but to ensure that $\sum v_i \ne 0$, the simplest way is to have $v_i = x_i^2$. Then $u_i = v_is_i =x_iy_i$.
Of course, while doing this my hands are waving so fast that I might take flight. But this is the best that I can do right now.
As I wrote at the start, it is really the fact that this is the way the math works out.