Subgaussian increments of excess risk for Lipschitz functions

180 Views Asked by At

So the problem I'm having trouble with is part (a) of exercise 8.4.9 in Vershynin's High Dimensional Probability book.

The setup is as follows. Let us define a family of functions $\mathcal{F}$ and a target function $T$ as follows:

$$\mathcal{F}=\{ f:[0,1]\rightarrow \mathbb{R}, ||f||_\text{Lip} \leq L\}$$

and a target function $T:[0,1]\rightarrow [0,1]$.

Let $X, X_1, \cdots, X_n$ be iid. Show that the random process

$$X_f = \frac{1}{n}\sum_{i=1}^n [(f(X_i)-T(X_i))^2-E(f(X)-T(X))^2]$$

where $f\in \mathcal{F}$, has subgussian increments:

$$ ||X_f-X_g||_{\psi_2} \leq \frac{CL}{\sqrt{n}}||f-g||_\infty $$

$C$ is an absolute constant.

For some reason I am having a really hard time trying to show this. I have already tried the usual tricks such as rewriting the expectation in terms of another iid variable $Y_i$ so that you can write

$$X_f = \frac{1}{n}\sum_{i=1}^n E[(f(X_i)-T(X_i))^2-(f(Y_i)-T(Y_i))^2]$$

And tried manipulating this expression around, but I always got stuck trying to bound

$$||f(X_i)^2-g(X_i)^2 - (f(Y_i)^2-g(Y_i)^2)||^2_{\psi_2}$$

But I couldn't get $L$ and $||f-g||_\infty$ terms to come out simultaneously.