Given a loss $E$: $$E = (Y - X\beta)^T(Y - X\beta) + \lambda\left \| \beta \right \|^2$$ The value of $\beta$ that minimizes the loss can be obtained by setting $\frac{\partial E}{\partial \beta} =0$
However I am interested in something like this:
$$E = (Y - X_1\beta - X_2\alpha)^T(Y - X_1\beta - X_2\alpha) + \lambda\left \|\alpha+ \beta \right \|^2$$
Can just solve for $\alpha$ and $\beta$ by setting $\frac{\partial E}{\partial \alpha} =0$ and $\frac{\partial E}{\partial \beta} =0$ so that I get two equations for two variables ? Or there is some other way?
Let $$\gamma = \begin{pmatrix} \alpha \\ \beta \end{pmatrix}$$ and define the matrices $$\Gamma = \begin{pmatrix} X_2 \; | \; X_1 \end{pmatrix}, \qquad \Lambda=\begin{pmatrix} \mathbb{I} \; | \; \mathbb{I} \ \end{pmatrix},$$ where $\mathbb{I}$ is the identity matrix with order equal to the length of $\alpha$ (or $\beta$). Then, you can write your expression as $$E = (Y - \Gamma\gamma)^\top (Y - \Gamma\gamma) + \lambda ||\Lambda\gamma||^2$$ and find the optimal values of $\gamma$ solving the equation $$\frac{dE}{d\gamma} = 0.$$