Conditions for $\beta_0=\bar{y}$ in simple linear regression using least squares

82 Views Asked by At

If we consider a linear model $\mathbf{y}=\beta_0\mathbf{1}+\beta_1\mathbf{a_1}+\beta_0\mathbf{a_2}+\varepsilon$ with $a_1,a_2$ column vectors with $n$ entries and $\beta_1\neq\beta_2\neq 0$.

Then, what would be the conditions on $a_1$ and $a_2$ for $\beta_0=\bar{y}$?

I've re writen the linear formula as $\mathbf{y}=\mathbf{X}\mathbf{\beta}+\mathbf{\varepsilon}$,

which gives $\mathbf{\beta}=(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T\mathbf{y},\quad$ with $\beta=\begin{pmatrix} \beta_0 \\ \beta_1 \\ \beta_2 \end{pmatrix}\quad$ and $\quad\mathbf{X}=\begin{pmatrix} 1 & a_1 & a_2\end{pmatrix}$.

However I don't know how to go from there, when I try to compute $(\mathbf{X}^T\mathbf{X})^{-1}$ it gives me an non inversible matrix.

Any help would be appreciated, thanks!

1

There are 1 best solutions below

0
On

If you want $\bar{y}=\beta_0$ then from the first equation you've set the condition that $\beta_1\mathbf{a}_1=-\beta_2\mathbf{a}_2$. Meaning that $\mathbf{a}_1,\mathbf{a}_2$ are linearly dependent.

However, for the last equation you formulated, $\mathbf{X}^T\mathbf{X}$ is invertible if and only if $\mathbf{X}$ is full column rank, meaning that $1, a_1, a_2$ are linearly independent.

Therefore you have conflicting conditions. To conclude, the answer to your original question is that your original data matrix needs to be of column rank 1, meaning that all variables are a simple linear transformation of each other. If this conditions is satisfied, then solving linear regression via least squares fails.