Bounding a deviation from the mean

95 Views Asked by At

Suppose $\xi_1,\ldots,\xi_n$ denote independent (but not necessarily identically distributed) random variables. Let $\mathcal{G}$ denote a finite set of $N$ functions and assume that for every $g\in \mathcal{G}$,

$$\sup_x |g(x)|\leq \beta \textrm{ and } \sum_{i=1}^n Eg^2(\xi_i)\leq \delta^2.$$

I'm trying to show that there exists a positive constant $C$ such that $$E \max_{g\in \mathcal{G}} \big| \sum_{i=1}^n (g(\xi_i)-Eg(\xi_i)) \big|\leq C \delta \sqrt{\log(2N)} \textrm{ if } \beta\leq \delta/\sqrt{\log(2N)}.$$ Does anybody have a suggestion for how to go about this? I'm stuck, honestly. I've been trying to use Jensen's inequality and the fact that $g(\xi_i)-E g(\xi_i)$ is subgaussian.

1

There are 1 best solutions below

0
On

The following calculation does, unfortunately, not lead to the estimate you are looking for, but perhaps it helps you in some way...


Let $\mathcal{G} := \{g_1,\ldots,g_N\}$ and

$$A_j := \left\{\left|\sum_{i=1}^n g_j(\xi_i)-\mathbb{E}g_j(\xi_i) \right| = \max_{k=1,\ldots,N} \left|\sum_{i=1}^n g_k(\xi_i)-\mathbb{E}g_k(\xi_i) \right| \right\}.$$

From the Cauchy Schwarz inequality we see that

$$\begin{align*} I:=\mathbb{E} \max_{k=1,\ldots,N} \left|\sum_{i=1}^n g_k(\xi_i)-\mathbb{E}g_k(\xi_i) \right| & = \sum_{j=1}^N \mathbb{E} \left( 1_{A_j} \left|\sum_{i=1}^n g_j(\xi_i)-\mathbb{E}g_j(\xi_i) \right| \right) \\ &\leq \sum_{j=1}^N \sqrt{\mathbb{P}(A_j)} \sqrt{\mathbb{E} \left( \left| \sum_{i=1}^n g_j(\xi_i)-\mathbb{E}g_j(\xi_i) \right|^2 \right)} \tag{1} \end{align*}$$

It follows from the independence of the random variables and the Jensen inequality that

$$\begin{align*} \mathbb{E} \left( \left| \sum_{i=1}^n g_j(\xi_i)-\mathbb{E}g_j(\xi_i) \right|^2 \right) &= \mathbb{E}\left( \sum_{i=1}^n |g_j(\xi_i)-\mathbb{E}g_j(\xi_i)|^2 \right) \\ &\leq 2\beta \log \exp \mathbb{E}\left( \sum_{i=1}^n |g_j(\xi_i)-\mathbb{E}g_j(\xi_i)| \right) \\ &\leq 2\beta \log\left[ \mathbb{E} \exp\left( \sum_{i=1}^n |g_j(\xi_i)-\mathbb{E}g_j(\xi_i)| \right) \right]. \tag{2} \end{align*}$$

Using again the independence of the random variables, we get

$$\begin{align*} \mathbb{E} \exp \left( \sum_{i=1}^n |g_j(\xi_i)-\mathbb{E}g_j(\xi_i)| \right) &= \prod_{i=1}^n \mathbb{E} \exp\big(|g_j(\xi_i)-\mathbb{E}g_j(\xi_i)| \big) \\ &= \prod_{i=1}^n \left[1+ \frac{1}{2} \mathbb{E}|g_j(\xi_i)-\mathbb{E}g_j(\xi_i)|^2 + \mathbb{E}\left(\sum_{k \geq 3} \frac{|g_j(\xi_i)-\mathbb{E}g_j(\xi_i)|^{k}}{k!} \right) \right] \\ &\leq \prod_{i=1}^n \left[1+ \mathbb{E}|g_j(\xi_i)-\mathbb{E}g_j(\xi_i)|^2 e^{2\beta} \right]\end{align*}$$

Plugging this estimate into $(2)$ and using that $\log(1+x) \leq x$ yields

$$\begin{align*} \mathbb{E} \left( \left| \sum_{i=1}^n g_j(\xi_i)-\mathbb{E}g_j(\xi_i) \right|^2 \right)&\leq 2\beta \sum_{i=1}^n \log\left[1+ \mathbb{E}|g_j(\xi_i)-\mathbb{E}g_j(\xi_i)|^2 e^{2\beta} \right] \\ &\leq 2\beta e^{2\beta} \sum_{i=1}^n \mathbb{E}|g_j(\xi_i)-\mathbb{E}g_j(\xi_i)|^2 \leq 2\beta \delta^2 e^{2\delta} \end{align*}.$$

Hence,

$$I \leq \sqrt{2\beta} \delta \sqrt{e^{2\beta}} \sum_{j=1}^N \sqrt{\mathbb{P}(A_j)}$$