uncertainty, error propagation, residuals and least squares.

259 Views Asked by At

I'm stuck with something, I haven't found a proper discussion about it. Say you have a set of data $\sum_i^n (x_i, y\pm \sigma_i) $ which follows a linear trend, where there's an uncertainty $\sigma_i$ in the y variable. Now in the least-squares method, we assume that the variable $y_i$ (one of the n variables) follows a normal distribution with standard deviation $\sigma_i$.

$P_i = c \space exp[-1/2 (y_i -\mu_i)^2 /\sigma_i^2] $

where c is some constant.

Now, $\mu_i $ is going to be the expected value of y, so $\mu_i \rightarrow \hat{y_i}$ , where $\hat{y_i}$ is the $y$ point in the fitted line, given $x_i$.Therefore, the numerator in the probability is the residual itself. Thus, the residuals also follow a normal distribution.

But here is my question, what's the relationship between the standard deviation $\sigma_i$ and the residuals, I mean, of course, they are related, but does it mean that the uncertainty in the $y$ variable is also the standard deviation of the residual? Shouldn't the standard deviation of the residual depends on the coefficients of the linear regression line by error propagation? I'm getting all confused about it. Thank you very much guys.