I want to minimise Least squares problem with second derivative regularisation, that is, minimizing the following w.r.t $\mathbf{g}(x)$. $$min_{\mathbf{g}(x)} ||\mathbf{A} \cdot \mathbf{g}(x) - \mathbf{b}||^2 + \lambda ||\mathbf{g}''(x)||^2.$$ Can someone show me how to solve this or point me to a solution?
Also, i found that this problem should be equivalent to normal least squares with modified matrix $$\begin{bmatrix}A\\ \sqrt{\lambda} D \end{bmatrix},$$ where D is second derivative matrix, i.e., $D = \begin{bmatrix}1, -2, 1,0 ...\\ 0, 1, -2, 1, 0,...\end{bmatrix}.$
I found it hard to prove this,, so i would be glad if you could explain the correspondence between two.
$ \def\l{{\sqrt\lambda}} \def\frbb#1{\left\| #1 \right\|_F^2} \def\m#1{\left[\begin{array}{c}#1\end{array}\right]} \def\mc#1{\left[\begin{array}{c|c}#1\end{array}\right]} $First note that you must modify both the $A$ matrix and the $b$ vector $$\eqalign{ \m{A\\ \l D}g \;-\; \m{b\\0} \;=\; \m{Ag-b\\ \l Dg} \\ }$$ Then note that the squared Frobenius norm of a block-partitioned matrix is the sum of the squared Frobenius norms of the individual blocks, i.e. $$\eqalign{ \frbb{ \mc{W&X\\\hline Y&Z} } &= \frbb W + \frbb X + \frbb Y + \frbb Z \\ }$$ Putting these two observations together yields $$\eqalign{ \frbb{\m{A \\ \l D}g \;-\; \m{b\\0}} \;=\; \frbb{Ag-b} \;+\; \frbb{\l Dg} \\ }$$ It is then trivial to pull $\lambda$ out of the norm in the last term and substitute $g'' = Dg$