I have a problem when reading through Theoretical Statistics by Robert. W. Kenner, Theorem 9.1 in Chapter 9, pp. 152-153. It is about the continuity of random functions.
By random functions, they are defined as:
Let $X_1, X_2, \dots$ i.i.d., and $K$ be a compact set in $\mathbb{R}^p$. Define $$ W_i(t) = h(t, X_i), \quad t \in K, $$ where $h$ is a continuous function of $t$ for all $x$. Then $W_1, W_2, \dots$ are i.i.d. random functions in $C(K)$, the space of continuous functions on $K$.
In my question, the randomness should not be so important, and I think it is safe to treat them as continuous functions on $C(K)$.
The theorem states that:
Theorem 9.1 Let $W$ be a random function in $C(k)$ and define $$ \mu(t) = \mathbb{E}W(t), \quad t \in K $$ ($\mathbb{E}$ is the expectation). If $\lvert \lvert W \rvert \rvert _\infty < \infty$ (the sup norm of $W$), then $\mu(t)$ is continuous. Also $$ \sup _{t \in K} \: \{ \mathbb{E} \sup _{s: \lvert \lvert s - t \rvert \rvert < \epsilon} |W(s) - W(t)| \} \to 0 \quad \text{as } \epsilon \to 0. $$
The part that I am struggling with is proof on the second part,
Define $$ M _\epsilon (t) = \sup _{s: \lvert \lvert s - t \rvert \rvert < \epsilon} |W(s) - W(t)|, $$ ... . $\color{red}{\text{Because $W$ is continuous, $M_\epsilon$ is continuous}}.$
I cannot figure out why this claim is true. If I write it out,
$$ |M_\epsilon(t_1) - M_\epsilon(t_2)| = \left| \sup _{s_1: \lvert \lvert s_1 - t_1 \rvert \rvert < \epsilon} |W(s_1) - W(t_1)| - \sup _{s_2: \lvert \lvert s_2 - t_2 \rvert \rvert < \epsilon} |W(s_2) - W(t_2)| \right|. $$
Usually the ineqaulity $\left| |x| - |y| \right| \leq |x - y|$ will be applied, but in this case there are two balls with different centers. Even though one can control the distance between $t_1$ and $t_2$, the distance between $s_1$ and $s_2$ cannot since it depends on $\epsilon$ which is a parameter of the function $M$.
I also tried considering only small $\epsilon$'s. A brief outline is suppose continuity of $W$ means for any $\gamma > 0$ there is $\delta > 0$ such that $$ \lvert \lvert t_1 - t_2 \rvert \rvert < \delta \implies |W(t_1) - W(t_2)| < \frac{\gamma}{2}, $$ then for $\epsilon < \delta$, $$ |M_\epsilon(t_1) - M_\epsilon(t_2)| \leq \gamma. $$ However, the choice of $\epsilon$ depends on $\gamma$, and to show continuity we need $\epsilon$ to be independent of $\gamma$.
Please share your thoughts about this problem. Any help is appreciated.
Edit: After reading through the comments from @Porufes, I come up with the following incomplete argument. The problem is simplified to 1D case, i.e. to show that
If $f$ is a continuous function on $K$, where $K \subset R$ is compact, then $$ d_\epsilon (t) = \sup _{x \in (t-\epsilon, t+\epsilon)} |f(x) - f(t)| $$ is continuous on $K$.
My argument: Let $\epsilon > 0$ be fixed. Suppose $d_\epsilon$ is not continuous at $x \in K$, i.e. there is $\gamma_0 > 0$, such that for all $\delta > 0$, there is $y$ which $|x - y| < \delta$ and $|d_\epsilon(x) - d_\epsilon(y)| > \gamma_0$.
We first pick $\delta > 0$ according to the uniform continuity of $f$, such that $$ x, y \in K, \: |x - y| < \delta \implies |f(x) - f(y)| < \frac{\gamma_0}{3}. $$
Let $\gamma > 0$ and $\epsilon > 0$ be fixed, and find $\delta > 0$ according to the uniform continuity of $f$, such that $$ x, y \in K, \: |x - y| < \delta \implies |f(x) - f(y)| < \gamma. $$ For any $y$ satisfying the requirements, the compactness of $K$ and continuity of $f$ implies the existence of $$ \begin{align*} p \in [x - \epsilon, x + \epsilon] \cap K, &\quad |f(x) - f(p)| = d_\epsilon(x), \\ q \in [y - \epsilon, y + \epsilon] \cap K, &\quad |f(y) - f(q)| = d_\epsilon(y). \end{align*} $$ The discontinuity at $x$ can be considered as two separate cases.
Case 1: $d_\epsilon (x) \geq \gamma_0 + d_\epsilon (y)$. By the choice of $\delta$, $$ \begin{align*} \big||f(x) - f(p)| - |f(y) - f(p)|\big| &\leq |f(x) - f(y)| < \frac{\gamma_0}{3} < \gamma_0 , \\ \implies |f(y) - f(p)| &> |f(x) - f(p)| - \gamma_0 \\ &= d_\epsilon(x) - \gamma_0 \\ &\geq d_\epsilon(y). \end{align*} $$ Because $d_\epsilon(y)$ is the supremum over $I = (y - \epsilon, y + \epsilon) \cap K$, the above implies that $p \not \in I$, or $p \in [x - \epsilon, y - \epsilon) \cap K$. $\color{red}{\text{Suppose that there is}}$ $$ \color{red} { z \in (p, p + \delta) \cap (y - \epsilon, y + \epsilon) \cap K, } $$ then since $|z - p| < \delta$, $$ |f(p) - f(z) | < \frac{\gamma_0}{3}. $$ Moreover, as $z \in (y - \epsilon, y + \epsilon)$ and $|f(y) - f(t)|$ attains maximum at $t = q$, $$ |f(y) - f(p)| \geq |f(y) - f(q)| \geq |f(y) - f(z)|. $$ Therefore $$ \begin{align*} | f(q) - f(y) | - | f(z) - f(y) | &\leq |f(p) - f(y)| - |f(z) - f(y)| \\ &\leq |f(p) - f(z)| \\ &< \frac{\gamma_0}{3}. \end{align*} $$ Hence $$ \begin{align*} |d_\epsilon(y) - d_\epsilon(x)| &= \big| |f(x) - f(p)| - |f(y) - f(q)| \big| \\ &\leq |f(x) - f(y) + f(p) - f(z) + f(z) - f(q)| \\ &\leq |f(x) - f(y)| + |f(p) - f(z)| + |f(z) - f(q)| \\ &< \frac{\gamma_0}{3} + \frac{\gamma_0}{3} + \frac{\gamma_0}{3} \\ &= \gamma_0. \end{align*} $$ This contradicts with $d_\epsilon (x) \geq \gamma_0 + d_\epsilon (y)$, thus the assumption does not hold.
For the second case $d_\epsilon (x) \leq d_\epsilon (y) - \gamma_0$, I have not worked out the detail but I assume it is similar to the first case.
The problem of this argument is that the $\color{red}{\text{line in red}}$ may not hold if $K$ is not an interval, since such $z$ may not exists.
Another insight I got is that the size of $\epsilon$ cannot be too large. For instance, if $K = [0, 1] \cup [2, 3]$, and $$ f = \begin{cases} 1 & \text{if } x \in [0, 1], \\ 0 & \text{if } x \in [2, 3]. \end{cases} $$ and $\epsilon = 1.5$, then $$ d _{1.5}(x) = \begin{cases} 0 & \text{if } x \in [0, 0.5] \cup (2.5, 3], \\ 1 & \text{if } x \in (0.5, 1] \cup [2, 2.5], \end{cases} $$ which is not continuous on $K$. Thus it is needed to limit $\epsilon$ so that it splits $K$ into, roughly speaking, "connected components" so that $f$ is "smooth enough".
Here are my thoughts after thinking about this for a week. Again, any help is appreciated.
Assume $W$ continuous $K\to R$ and first assume $K$ convex. Let $M_\epsilon(t) = \sup_{s:|t-s|\le \epsilon} |W(t)-W(s)|$. Since the intersection of $K$ with the closed ball around $t$ is compact, there is $s_t$ such that $M_\epsilon(t)=|W(t)-W(s_t)|$. Now for $x,t$, $$ M_\epsilon(t) - M_\epsilon(x) \le |W(t)-W(s_t)| - |W(x)-W(s_x)| \le |W(t)-W(s_t)|-|W(x)-W(S)| $$ where we define $S$ to be $S=P_K(x+(s_t-t))$ where $P_K$ is the convex projection onto $K$. By the triangle inequality, $$ M_\epsilon(t) - M_\epsilon(x) \le |W(t)-W(x) + W(S)-W(s_t)|\le |W(s_t)-W(S)| + |W(t)-W(x)|. $$ By Heine's theorem, for any $e>0$ there exists $\delta_e$ such that $|a-b|\le \delta \Rightarrow |W(a)-W(b)|\le e$. If $|t-x|\le \delta$ then $|W(t)-W(x)|$ and (drawing a parallelogram picture helps) we have because $P_K$ is 1-Lipschitz $$ |S-s_t| = |P_K(x+(s_t-t)) - O_K(s_t)| \le |x+(s_t-t)-s_t|=|x-t|\le \delta $$ and $|W(s_t)-W(S)|\le e$ as well.
Now assume that for any $\bar K$ convex compact and any continuous $\bar h:K\times R\to R$, using what we did above thanks to convexity of $\bar K$, we can prove that $$ \sup_{t\in \bar K} E[\sup_{s\in \bar K: |s-t|\le \epsilon} |\bar W(s)-\bar W(t)|] \to 0 $$ as $\epsilon\to 0$, where $\bar W_i(u) = h(u,X_i)$.
If $K$ is compact but not convex and $h:K\times R\to R$ is continuous, we can always find an extension $\bar h:R^p\times R\to R$ that is continuous and is an extension, in the sense $\bar h(t,x)= h(t,x)$ for all $t\in K, x\in R$ (see https://en.wikipedia.org/wiki/Tietze_extension_theorem). Now choose $\bar K$ to be a large enough compact and convex set (for instance a large Euclidean ball) such that $K\subset \bar K$. If thanks to convexity you are able to prove $$ \sup_{t\in \bar K} E[\sup_{s\in \bar K: |s-t|\le \epsilon} |\bar W(s)-\bar W(t)|] \to 0, $$ this implies $\sup_{t\in K} E[\sup_{s\in K: |s-t|\le \epsilon} |W(s)-W(t)|]\to 0$ as well as desired, because the suprema are taken over smaller sets and $W(s)=\bar W(s),\bar W(t)=W(t)$ for $s,t\in K$.