least squares regularization

Question

I am stuck at a numerical algebra question

For $\lambda\geq0$

$\displaystyle y(λ) = \arg \min_{y\epsilon R_n} ||b-Xy||_2^2 + \lambda||Ly||^2_2$

Show that if $\lambda_1>\lambda_2\geq 0$

$||Ly(\lambda_1)||_2 \leq ||Ly(\lambda_2)||_2$

I am trying to solve this using the fact that

$||b-Xy||^2_2 + \lambda_1||Ly||_2^2 = ||b-Xy||^2_2 + \lambda_2||Ly||_2^2 + (\lambda_1 - \lambda_2)||Ly||_2^2$

But how should I combine $y(\lambda_1)$ and $y(\lambda_2)$?

Answer 1

Because $y(\lambda_1)$ is the argmin of the $\lambda_1$ problem, we get: $||b-Xy(\lambda_1)||_2^2 + \lambda_1||Ly(\lambda_1)||_2^2 \leq ||b-Xy(\lambda_2)||_2^2 + \lambda_1||Ly(\lambda_2)||_2^2$

Because $y(\lambda_2)$ is the argmin of the $\lambda_1$ problem, we get $||b-Xy(\lambda_2)||_2^2 + \lambda_2||Ly(\lambda_2)||_2^2 \leq ||b-Xy(\lambda_1)||_2^2 + \lambda_2||Ly(\lambda_1)||_2^2$

Now add the two equations together and subtract the $b-Xy$ terms. We get $||Ly(\lambda_1)||_2^2(\lambda_1-\lambda_2) \leq ||Ly(\lambda_2)||_2^2(\lambda_1-\lambda_2)$. Using $\lambda_1> \lambda_2$ we get the result.