Tiny bandwidth function cannot be approximated by a combination of high bandwidth functions

30 Views Asked by At

Let $s > 0$ be a real number. Consider the space $D_s$ of the functions from $\mathbb{R}^k$ to $\mathbb{R}$ of the form $$ f(x) = \sum_{i=1}^{m} \alpha_i \exp\left( -\dfrac{\|x-x_i\|^2}{2s^2}\right) $$ where $m$ varies over positive integers, $\alpha_1,\dots,\alpha_m > 0$ and $x_1,\dots, x_m \in \mathbb{R}^k.$

These sort of functions come up in various machine learning techniques.

Informally as $s \to 0^+$ the function $\exp\left( -\dfrac{\|x-x_i\|^2}{2s^2}\right)$ tends to the function that is $1$ at $x_i$ and is $0$ elsewhere and so gets more spiky, similarly as $s$ increases $\exp\left( -\dfrac{\|x-x_i\|^2}{2s^2}\right)$ gets flatter and tends to the function that is 1 everywhere.

It appears intuitive that if $0 < s_1 < s_2$, and if $s_1$ is sufficiently smaller than $s_2$, in some sense, then any function of the form $\exp\left( -\dfrac{\|x-x_0\|^2}{2s_1^2}\right)$ cannot be well approximated by functions in $D_{s_2}$, in some sense. My purpose in asking this questions is to request for references about such a result, if it is true, or an argument why this is not true.

I recall seeing a result of this type in a thesis online, but I cannot find it anymore.