QR factorization for ridge regression

Question

I am solving an overdetermined system of equations: $$Ax= b$$

Using QR factorization, we can solve this system easily by posing it as:

$$Rx= Q'b$$

I would like to regularize my estimate of $x$. I can use ridge regression with the normal equations being:

$$(A'A+\lambda\Gamma'\Gamma)x=A'b$$

My question is, how can I directly transform $A$ to matrix $\Delta$ such that solving the system $\Delta x=b$ via QR factorization yields the regularized estimate for $x$.

Answer 1

Try writing it as (using $\lambda = 1$ for display simplicity) $$B = \pmatrix{ A \\ \Gamma}$$ then you have for the equation $Bx = \pmatrix{b \\ 0}$

\begin{align} B'Bx &= B' \pmatrix{b \\ 0} \\ \pmatrix{ A' & \Gamma'}\pmatrix{ A \\ \Gamma}x &= \pmatrix{ A' & \Gamma'} \pmatrix{b \\ 0} \\ (A'A + \Gamma'\Gamma)x &= A'b \end{align}

Therefore in terms of the QR factorization, take the QR factors of $$B = \pmatrix{ A \\ \Gamma}$$