DKW-style $\ell_{\infty}$ bounds for sum of i.i.d. random functions: $\to [0,1]$

136 Views Asked by At

Let $\mathbf{G}$ be the set of (edit: convex) functions $g: X \to [0,1]$, where $X$ is a compact subset of $\mathbb{R}^d$ or something like that.

Suppose I have a distribution $D$ on $\mathbf{G}$. Let $\bar{g}: X \to [0,1]$ be the pointwise average $$ \bar{g}(x) = \mathbb{E}_{g \sim D} g(x) . $$

When I draw $n$ functions $g_1,\dots,g_n$ from $D$ i.i.d., let $\hat{g}_n$ be a random variable equalling the empirical pointwise average, $$ \hat{g}_n(x) = \frac{1}{n} \sum_{j=1}^n g_j(x) . $$

Recall that $\| f \|_{\infty} = \sup_{x \in X} |f(x)|$. I'd ideally like a bound of this sort

$$ \Pr\left[ \| \hat{g}_n - \bar{g} \|_{\infty} > \epsilon \right] ~~ \leq ~~ ??? $$

Are there known inequalities of this form? I'd naively hope for something like $O(e^{-n\epsilon^2})$. If it's not true, it'd be great to know conditions that make it true (all functions are continuous, Lipschitz, ...?).

(Edit: (1) I should mention the related DKW inequality that inspires this goal. (2) Changed the condition on functions to be convex, due to ClementC.'s counterexample below.)

2

There are 2 best solutions below

0
On BEST ANSWER

This question was asked and answered on MO: https://mathoverflow.net/questions/171527/concentration-inequalities-in-ell-infty-for-sums-of-iid-random-nice-fu

The keyword for this type of problem is "empirical process", and references exist.

3
On

A small change ("against" continuous function -- not convex):

Fix any $n>1$, and consider the distribution $D_n$ supported on all piecewise-affine functions of the form $g_y\colon x\in [0,1]\mapsto g_y(x)$, where $$ g_y(x)=\begin{cases} 1 & \text{if }x=y\\ 0 & \text{if }\lvert x-y \rvert \geq \frac{1}{2n^2}\\ \text{affine otherwise} \end{cases} $$ $D_n$ is defined by drawing $y\in [0,1]$ uniformly at random, and outputting the continuous function $g_y$. Then $\bar{g}(x)\stackrel{\rm def}{=}\mathbb{E}_{y\sim \mathcal{U}_{[0,1]}}[ g_y(x)] \leq \frac{1}{n^2}$ (as there are at most a set of measure $1/n^2$ of functions $g_y$ which are non-zero on $x$), but for any outcome of the $n$ draws $y_1,\dots,y_n$ (taking for example $x=y_1$, which implies $\hat{g}_n(x) = \frac{1}{n}\sum_{j=1}^n g_{y_j}(x) \geq \frac{1}{n}$) you have $$ \lVert\hat{g}_n - \bar{g} \rVert_\infty \geq \frac{1}{n}-\frac{1}{n^2} \geq \frac{1}{2n} $$ with probability $1$. So for $\epsilon < \frac{1}{2n}$, there cannot be any non-trivial bound of the sort you seek.