It is clear that the four assumptions of a linear regression model are: Linearity, Independence of error, Homoscedasticity and Normality of error distribution. My question is does any of these four assumption imply all the Xs are independent to each other (aka X is full column rank)?
The four assumptions on linear regression
589 Views Asked by Jolly_8878 https://math.techqa.club/user/jolly-8878/detail AtThere are 2 best solutions below
No and also independence of the $X_i$'s (row vector, i-th row of design matrix) does not have anything to do with full rank either. For the design matrix you need additional assumptions. There are two ways in which people usually proceed:
1) Fixed (deterministic) design: Since it is deterministic, it either has full rank or it does not, so you just have to assume it!
2) Random design: For asymptotic considerations here one usually has to assume that $\frac{1}{n}\sum_{i=1}^n{X_i^TX_i} \to C$ in probability, where $C$ is a full rank (invertible) matrix. Here you can make a connection with independence: If the $X_i$'s are i.i.d., then by the law of large numbers you just need that $ \mathbb E[X_i^TX_i]$ is full rank.
No, none of these assumptions imply the full rank condition on $X$. It has to be assumed separately. The full rank condition on $X$ does not mean independence of regressors from each other -- it just means that they are not related by a linear functional relationship.
Independence of the error can be relaxed (to uncorrelatedness), and homoschedasticity and normality can be relaxed as well.