Suppose we want to estimate $\beta$ by minimizing $L(\beta)=\sum_{i=1}^n(y_i-\beta x_i)^2+\lambda|\beta|$, where $\lambda$ is a fixed positive constant. Calculate the estimate.
How would I calculate $\beta$ in this scenario?
where the linear regression model is $$y_i = \beta x_i + \epsilon _i; i = 1,..., n$$
Consider first $\beta>0$. First order condition with respect to $\beta$ would imply $$\frac{\partial L(\beta)}{\partial\beta}=0 \Rightarrow -\sum^{n}_{i=1} 2(y_i-\beta x_i)x_i +\lambda=0\Rightarrow \sum^{n}_{i=1}(y_i-\beta x_i)x_i=\lambda/2\Rightarrow \beta=\frac{-\lambda/2+\sum^{n}_{i=1}y_ix_i}{\sum^{n}_{i=1}x^2_i}$$ So your optimal estimate is $$\beta^*=\frac{-\lambda/2+\sum^{n}_{i=1}y_ix_i}{\sum^{n}_{i=1}x^2_i}$$ if $\beta<0$ then $$\beta^*=\frac{\lambda/2+\sum^{n}_{i=1}y_ix_i}{\sum^{n}_{i=1}x^2_i}$$