Confusion over L2-norm polynomial approximation

353 Views Asked by At

Q1: Let $C(\mathcal{X})$ be the space of continuous functions $f:[a,b]\to\mathbb{R}$. Let $F(\mathcal{X})$ be a subset of $C(\mathcal{X})$ that is dense in $C(\mathcal{X})$ w.r.t. metric $d_{\infty}(\cdot,\cdot)$. Is $B=\{f\in F(\mathcal{X}): \lVert{f}\rVert^2\leq 1\}$ dense in $A=\{f\in C(\mathcal{X}): \lVert{f}\rVert^2\leq 1\}$? ($\lVert{\cdot}\rVert^2$ is L2-norm.)

Q2: Assume the answer for Q1 is yes. Let $B,A$ be defined as Q1. For two points $x_1,x_2\in [a,b]$, we have \begin{align*} \sup_{f\in A} \lvert f(x_1)-f(x_2)\rvert =\sup_{f\in B} \lvert f(x_1)-f(x_2) \rvert.\tag{*} \end{align*}

Let $F(\mathcal{X})=\overline{\mathrm{span}}\{k_x: x\in[a,b]\}$, where $k_x$ is a feature map induced by a Gaussian kernel $\langle k_x,k_y\rangle=k(x,y)=\mathrm{e}^{{-\alpha(x-y)}^2}$. Then $F(\mathcal{X})$ is a RKHS and it has been proven that it is dense in $C(\mathcal{X}$) for any positive constant $\alpha$. Then I get: \begin{align*} \sup_{f\in B} \lvert f(x_1)-f(x_2) \rvert &=\sup_{f\in B} \langle k_{x_1}-k_{x_2}, f\rangle\\ &=\sqrt{\langle k_{x_1}-k_{x_2},\ k_{x_1}-k_{x_2}\rangle}\quad\text{ (**)}\\ &=\sqrt{2-2\mathrm{e}^{{-\alpha(x_1-x_2)}^2}}. \end{align*} Now the result depends on $\alpha$. It is expected to get a constant value for different $\alpha$. What's wrong with the process?

1

There are 1 best solutions below

2
On BEST ANSWER

I think you mix up the two different(!) norms $\lVert f\rVert_{L^2}$ and the norm from the Gaussian Kernel $\lVert f\rVert_{\exp}$. Your equation (*) assumes you work with the unit balls of the $L^2$-norm while (**) is only true for the unit ball in the RKHS norm.

With respect to (**): Let $\langle\cdot,\cdot\rangle_{\exp}$ be the scalar product of the Gaussian RKHS. Then the equation should read/is only true for: $$ \sup_{\lVert f\rVert_{\exp}\leq 1} \langle k_{x_1}-k_{x_2}, f\rangle_{\exp} =\sqrt{\langle k_{x_1}-k_{x_2},\ k_{x_1}-k_{x_2}\rangle_{\exp}}.$$

And since $\lVert f\rVert_{L^2} \neq \lVert f\rVert_{\exp}$ the unit balls $B= \{f\mid \lVert f\rVert_{L^2} \leq 1\}$ and $\{f\mid \lVert f\rVert_{\exp} \leq 1\}$ are not equal. That is why $$ \sup_{f\in B} \langle k_{x_1}-k_{x_2}, f\rangle_{\exp} \neq \sqrt{\langle k_{x_1}-k_{x_2},\ k_{x_1}-k_{x_2}\rangle_{\exp}}.$$

(And $\alpha$ has to be negative)