Find CI for mean of linear regression with variance unknown

Question

For the simple linear regression model:

$Y_i = \beta*X_i + \epsilon_i$

I want to find CI for $\beta x$ which is $E(Y_i)$ when $x_i=x$

I find that $\hat\beta$ ~ $N(\beta, \frac{\sigma^2}{\sum(X_i^2)})$, so the distribution of $\hat\beta x$ is $N(\beta x$, $\frac{X^2\sigma^2}{\sum(X_i^2)})$. If the variance is known I can use $P(-z_{\alpha/2}$ $\leq$ $\frac{\hat\beta x - \beta x}{x\sigma/\sum(x_i^2)}$ $\leq$ $z_{\alpha/2})$.

If the variance is unknown, what unbiased estimator should I use for $\sigma^2$? Is it the sample variance? What is it in this case?

Answer 1

Suppose number of predictors is $p$ and number of observations is $N$. Let $X$ be $N\times p$ matrix, and the specific $x$ be $1\times p$. Then $$\frac{x\beta-x\hat{\beta}}{\hat{\sigma}\sqrt{x(X^TX)^{-1}x}} \sim t_{N-p}$$ where $$\hat{\sigma}^2 = \frac{RSS}{N-p}=\frac{(Y-X\hat{\beta})^T(Y-X\hat{\beta})}{N-p}.$$

In the above $t$ stands for t distribution.

For simple regression $p=1$, so $\hat{\sigma}^2$ turns out to be sample variance. However, you need to use $t$ distribution, not Gaussian.

Answer 2

If your model is $Y_i = \beta x_i + \epsilon_i$, then the unbiased estimator for $var(\epsilon_i)=\sigma^2$ is given by $s^2 = \frac{\sum e_i^2}{n-1}$, where $e_i= y_i - \hat{y}_i$.