Is a $t$-statistic still $t$-distributed under linearly transforming the original OLS model?

67 Views Asked by At

For an OLS model $y=X\beta +\epsilon$ it is well known that any of the estimated coefficients, say $\widehat\beta_i$, divided by its estimated standard error $s_{\widehat\beta_i}$, must follow a $t$-distribution whose degree of freedom equal is $n-p$ where $p$ is the number of explanatory variables (including intercept).

Now, suppose we linearly transform the model by a non-singular matrix $C$: $$ \begin{align} y &= X\beta + \epsilon\\ &=XC C^{-1}\beta + \epsilon\\ &= XC\color{red}{\gamma} + \epsilon \end{align} $$ Then in this new model where the coeffcients to find are $\gamma$, we can find an OLS estimator $\widehat\gamma$ which is again a BLUE. But the question is, is there any way we can tell about the distribution of $\widehat\gamma$'s $t$-statistic $\widehat\gamma_i/s_{\widehat\gamma_i}$? For example, I believe it is still $t$-distributed, but how can we prove it? And what will be its dof?

Thanks!

1

There are 1 best solutions below

2
On BEST ANSWER

The (exact) $t$ distribution arises not from the OLS itself but from the combination of OLS, multivariate normal vector $\epsilon$ with covariance matrix of $\sigma ^ 2I$, and by assuming that the null hypothesis of $\gamma_i = 0$ is true.

In such a case, the OLS vector of $\hat{\gamma}$ or $\hat{\beta}$ is a linear transformation of $y$, i.e., $\hat{\beta} = (X'X)^{-1}X'y$, hence it is multivariate by itself and the marginal distribution of each estimator $\hat{\beta}_i$ is $N(0, \sigma_{i}^2)$. The residuals vector $e$ is also a linear transformation of $y$, i.e., $(I - H)y$, hence it is also follows a multivariate distribution and then you can show that $\hat{\sigma}_i^2 \sim \sigma_i ^ 2\chi^2(n-p)$, hence the ratio $$ \hat{\beta}_i/\hat{\sigma}_i $$ by definition follows $t$ distribution with $n-p$ dof. The explicit derivations can be found on wikipedia or any standard book on linear regression.

Having said that, regressing on $XC$ and $\gamma = X^{-1}\beta$ induces no change to this properties. Namely, you transformed the data $X$ and accordingly the coefficients vector $\beta$, you induced no change to the assumptions on $\epsilon$ or the estimation method (OLS), hence same properties will follow from the same reasoning. The only difference is that the BLUE property is now w.r.t. $\gamma$ given $XC$ and not w.r.t. $\beta$ given $X$.

If no normality assumption on $\epsilon$ is made then the normal distribution of $\hat{\beta}_i$ is merely an approximation for any finite sample size $n$, hence I would not bother at all with student's $t$ and will stick to the normal approximation.