I am trying to solve a modified least squares problem:
$$ \underset{x\epsilon\mathbb{R} }{min} \left \| Ax - b \right \|_{2}^{2} + \left \| Lx \right \|_{2}^{2} $$
To show that the solution is:
$$(A^{T}A + L^{T}L ) x = A^{T}b$$
So far I have this:
$$(Ax-b)^{T}(Ax-b) + (Lx)^{T}(Lx) $$
$$= x^{T}A^{T}Ax - x^{T}A^{T}b - b^{T}Ax + b^{T}b + x^{T}L^{T}Lx$$
$$= x^{T}(A^{T}Ax - A^{T}b + L^{T}Lx) - b^{T}(Ax-b)$$
I seem to obtain the solution provided the term in the first bracket is 0.
However, I am not sure if this is the correct approach to solving this least squares problem. I also don't know why the remaining $ b^{T}(Ax-b)$ term appears to be neglected?
For $$ f(x) = \lVert A x - b \rVert^2 + \lVert Lx \rVert^2 = \sum_{i=1}^n (Ax - b)_i^2 + (Lx)_i^2 $$ we get the $x$-gradient components \begin{align} (\partial/\partial x_j) f(x) &= \sum_{i=1}^n 2 (A x - b)_i a_{ij} + 2 (Lx)_i l_{ij} \\ &= 2 \left( \sum_{i=1}^n a^T_{ji}(A x - b)_i + l^T_{ji} (Lx)_i \right) \end{align} and for a critical point $$ A^T A x - A^Tb + L^T L x = 0 \iff \\ (A^T A + L^T L) x = A^T b \iff \\ x = (A^T A + L^T L)^{-1} A^T b $$