Questions on the Weierstrass approximation theorem

Question

From Williams' Probability w/ Martingales:

$\bf 7.3.\ $ Chebyschev's inequality

As you know this says that for $c\geq0$, and $X\in{\cal L}^2$, $$c^2\mathbf{P}(|X-\mu|>c)\leq \operatorname{Var}(X),\qquad\mu:=\mathbf{E}(X);$$ and it is obvious.

Example. Consider a sequence $(X_n)$ of IID RVs with values in $\{0,1\}$ with $$p=\mathbf P(X_n=1)=1-\mathbf P(X_n=0).$$

---

Then ${\bf E}(X_n)=p$ and $\operatorname{Var}(X_n)=p(1-p)\leq\tfrac14$. Thus (using Theorem $7.1$) $$S_n:=X_1+X_2+\cdots+X_n$$ has expectation $np$ and variance $np(1-p)\leq n/4$, and we have $${\bf E}(n^{-1}S_n)=p,\quad\operatorname{Var}(n^{-1}S_n)=n^{-2} \operatorname{Var}(S_n)\leq1/(4n).$$ Chebyschev's inequality yields $${\bf P}(|n^{-1}S_n-p|>\delta)\leq {1/(4n\delta^2)}.$$

---

$\bf 7.4.$ Weierstrass approximation theorem

If $f$ is a continuous function on $[0,1]$ and $\varepsilon>0$, then there exists a polynomial $B$ such that $$\sup_{x\in[0,1]}|B(x)-f(x)|\leq\varepsilon$$.

Proof. Let $(X_k)$, $S_n$ etc. be as in the Example in Section $7.3.$ You are well aware that $${\bf P}[S_n=k]=\binom{n\\k}p^k(1-p)^{n-k},\quad0\leq k\leq n.$$ Hence $$B_n(p):={\bf E} f(n^{-1}S_n)=\sum_{k=0}^n f(n^{-1}k)\binom{n \\k} p^k(1-p)^{n-k},$$ the '$B$' being in deference to Bernstein.

$\quad$ $\color{red}{\boxed{\color{black}{\text{Now }f\text{ is }\textit{bounded }\text{on }[0,1],}}}$ $|f(y)|\leq K,\forall y\in[0,1]$. Also, $\color{green}{\boxed{\color{black}{f\text{ is }\textit{uniformly continuous }\text{on }[0,1]:}}}$ for our given $\varepsilon>0$, there exists $\delta>0$ such that $$\tag{a}|x-y|\color{yellow}{\boxed{\color{black}{\leq}}}\delta\text{ implies that }|f(x)-f(y)|\leq \tfrac12\varepsilon.$$ Now, for $p\in[0,1]$, $$|B_n(p)-f(p)|=|{\bf E}\{f(n^{-1}S_n)-f(p)\}|.$$ Let us write $Y_n:=|f(n^{-1}S_n)-f(p)|$ and $Z_n:=|n^{-1}S_n-p|$. Then $Z_n\leq\delta$ implies that $Y_n<\tfrac12\varepsilon$, and we have $$\begin{align} |B_n(p)-f(p)|&\leq {\bf E}(Y_n) \\ & = {\bf E}(Y_n;Z_n\leq\delta)+{\bf E}(Y_n;Z_n>\delta)\\ & \leq \tfrac12\varepsilon{\bf P}(Z_n\leq\delta)+2K{\bf P}(Z_n>\delta) \\ & \leq\tfrac12\varepsilon+2K/(4n\delta^2). \end{align}$$ Earlier, we chose a fixed $\delta$ at $(\rm a)$. We now choose $n$ so that $$2K/(4n\delta^2)<\tfrac12\varepsilon.$$ Then $|B_n(p)-f(p)|<\varepsilon$, for all $p$ in $[0,1]$. $$\tag*{$\square$}$$

1. Red box: Is that because $f$ is continuous and defined on $[0,1]$? Is there a name for such fact (continuous and defined on closed interval $\implies$ bounded on said interval)?

2. Green box: Is that because $f$ is continuous and bounded on $[0,1]$? Is there a name for such fact (continuous and bounded on closed interval $\implies$ uniform continuous on said interval)?

3. Yellow box: Does it make a difference if this is $\le$ or $<$ ?

---

Answer 1

1. and 2. are well known properties of continuous functions on compact intervals, or more generally, on compact sets. If $f\colon[a,b]\to\mathbb{R}$ is continuous, then:

   • It is bounded and attains the maximum and the minimum (Weierstrass name is associated with this result.)
   • It is uniformly continuous

As for your last question, there is no difference between $<$ or $\le$. The proof can be written with any of them.