How to find the closed form formula for $\hat{\beta}$ using sparse least squares estimation?

49 Views Asked by At

I was given a linear regression model $$y = X\beta+u$$ with $u_i$ being i.i.d. of mean 0 and variance $\sigma^2$, $X \in R^{m\times n}$. $\hat\beta_s$ is the sparse least squares estimate of $\beta$ and $$\hat\beta_s=argmin\displaystyle\|Y-X\beta \|_{2}$$ subject to $\beta_1=0$ (first component of $\beta$ is 0). I thought about finding the vector of residual e and I suppose it's in this form: $$\mathbf{u}=\begin{bmatrix} \sqrt{(y_1 - \mathbf{x}_1\boldsymbol\beta_1)^2} \\ \sqrt{(y_2 - \mathbf{x}_2\boldsymbol\beta_2)^2} \\ \vdots \\ \sqrt{(y_n - \mathbf{x}_n\boldsymbol\beta_n)^2} \end{bmatrix}$$ with the 1st entry equals 0. Then I guess I should find $RSS=u^Tu$ and it's partial derivative, set it to 0 and find $\hat\beta$. Is my formula of $\mathbf{u}$ correct? If so, how should I expand the partial of RSS? Also if there's an ordinary LS $\hat\beta_{ls}$, can I assume its 1st component to be 0 as well?