I'm new to machine learning and currently I'm working on simple linear regression model. But I'm in trouble with computing the squared error. I have $$y=f(x;\theta)=(x,1)\theta=\theta_1 x+ \theta_2$$ I want to compute the minimum of $$L(\theta,\lambda)=\frac{1}{2} \sum_{i=1} ^2 (y_i-f(x_i;\theta))^2 +\frac{\lambda}{2}(\theta_1 ^2 + \theta_2 ^2) ~~(\lambda ≥ 0)$$ To minimize this, I have to solve $\frac{\partial}{\partial \theta}L(\theta,\lambda)$. But how should I compute this especially the part of $\frac{\partial}{\partial \theta}f(x_i;\theta)$. Should I separate $\theta$ into $\theta_1$ and $\theta_2$ then derivative? I would appreciate it if somebody help me to solve this. Thank you in advance.
[addition] And how should I compute the $$\lim_{\lambda \to \infty}\theta_\lambda ~~~?$$ I'm guessing I should compute the argmin of $\theta$ but I'm not pretty sure. What should I do to compute tthis?
For starters, your loss function is dependent on both $\theta_1, \theta_2$. So let's begin that way.
$$\begin{align*} \frac{\partial L(\theta_1, \theta_2, \lambda)}{\partial \theta_1} &= \frac{\partial\frac{1}{2} \sum_{i=1} ^2 (y_i-f(x_i,\theta_1,\theta_2))^2 +\frac{\lambda}{2}(\theta_1 ^2 + \theta_2 ^2)}{\partial \theta_1} \\ &=\frac{1}{2}\frac{\partial \sum_{i=1} ^2 (y_i-f(x_i,\theta_1,\theta_2))^2 }{\partial \theta_1}+\lambda\theta_1\\ &=\frac{1}{2}\sum_{i=1} ^2\frac{\partial (y_i-f(x_i,\theta_1,\theta_2))^2 }{\partial \theta_1}+\lambda\theta_1\\ &=\frac{1}{2}\sum_{i=1} ^2 2\cdot(y_i-f(x_i,\theta_1,\theta_2))\cdot\frac{\partial f(x_i,\theta_1,\theta_2) }{\partial \theta_1}+\lambda\theta_1\\ &=\frac{1}{2}\sum_{i=1} ^2 2\cdot(y_i-(\theta_1 x_i+ \theta_2))\cdot\frac{\partial (\theta_1 x_i+ \theta_2) }{\partial \theta_1}+\lambda\theta_1\\ &=\sum_{i=1} ^2 (y_i-(\theta_1 x_i+ \theta_2))\cdot x_i+\lambda\theta_1\\ \end{align*} $$
Similarly, you'd find that
$$\frac{\partial L(\theta_1, \theta_2, \lambda)}{\partial \theta_2} = \sum_{i=1} ^2 (y_i-(\theta_1 x_i+ \theta_2))\cdot 1+\lambda\theta_2$$