least square estimates to coefficient of analytic regression curve

Question

This is a bit of an embarrassing question because I thought I knew much (and have been using linear regression since forever) about regression until I read something in a text that challenged this.

For a linear regression curve Y on X, the mean of the distribution of the empirical $Y's$ is given by $\hat{Y} = \alpha + \beta x$. The $Y's$ have the form $Y = \alpha + \beta x + \epsilon$ where $\epsilon$ is (usually) standard normally distributed - arising from experimental variability.

Recall that for a set of $n$ observations/ empirical results demonstrating linearity, optimality of a regression curve fit to this set of observation is equivalent to obtaining the optimal coefficient $\alpha$, $\beta$ - and this is achieved by minimising the function

$\sum_{i=1}^{n}(y - \hat{y_{i}})^{2} = \sum_{i=1}^{n}(y - (\alpha + b x_{i}))^{2}$.

In a text, the author claims that $a, b$ are estimates of $\alpha, \beta$. I've never drawn any distinction between $a, b$ and $\alpha, beta$ prior to having read this.

Would be great if someone could shed some light on this.

Answer 1

$a$ and $b$ are the coefficients you obtain after performing linear regression on the data, which you can do regardless of what the data look like. in certain situations you may believe the data follow some linear relationship $Y = \alpha + \beta x + \epsilon$, where here $\alpha$ and $\beta$ are some "true" but unknown parameters that you would like to estimate. It is then prudent to ask whether the coefficients $a$ and $b$ you obtained from the linear regression are close to the "true" parameters $\alpha$ and $\beta$.

Answer 2

$\alpha$ and $\beta$ are "true" and unobservable values which describe the nature of relationship between $X$ and $Y$. As I mentioned, those are unobservable, however, you need to measure the relationship between $X$ and $Y$ somehow. Therefore, you try to estimate $\alpha$ and $\beta$. By applying, for instance OLS, you estimate $a$ for $\alpha$, and $b$ for $\beta$. According Gauss-Markov Theorem, if certain assumptions hold, then you can assume that $a$ and $b$ are the "best" (or closest) estimaters for $\alpha$ and $\beta$, respectively. But again, remember, you never observe "true" parameters, just you try to find "closest" estimaters $a$ and $b$.