The Hilbert norms of KRR solutions

42 Views Asked by At

Assume that we solve a kernel ridge regression problem for data set $X$ of size $n$ $$ \hat{g} = \mathrm{arg} \min_g \sum_{i=1}^{n}(y_i - g(x_i))^2 + ||g||^2_{\mathcal{H}} $$ and for extended dataset $X_*$ of size $n+m$, that includes $X$ $$ \hat{g}_* = \mathrm{arg} \min_g \sum_{i=1}^{n+m}(y_i - g(x_i))^2 + ||g||^2_{\mathcal{H}} $$ How to prove that? $$ ||\hat{g}||^2_{\mathcal{H}} \leq || \hat{g}_*||^2_{\mathcal{H}} $$