Understanding the proof, how to show that a sigmoidal function is a discriminatory

40 Views Asked by At

This is a part of the proof for the universal approximation theorem, which I already mentioned in another question. I have the outline of a proof but need help understanding it:

So I have $\varrho \in L_{loc}^{\infty}(\mathbb{R})$ and $\varrho$ is a sigmoidal function, meaning $\varrho(x) = \begin{cases} 1 & x \rightarrow \infty \\ 0 & x \rightarrow -\infty \end{cases} $ and need to show it's discriminatory. That means that for a signed Borel measure $\mu$ with \begin{equation} \int_{K}\varrho(\langle w, \ x\rangle + b) \,d\mu(x)=0 \ \ \ \forall w \in \mathbb{R}^d \text{ and } b \in \mathbb{R}, \end{equation} it implies $\mu=0$.

Now I already have: We know $\varrho( \langle w, \ x\rangle +b) \rightarrow \mathbb{1}_{(0, \ \infty)}(x)+\varrho(b)\mathbb{1}_{\{0\}}(x)$ for $w \rightarrow \infty$, because of the properties of $\varrho$. The proof now states, that with superposition and passing to the limit we get $\forall c_1, c_2 \in \mathbb{R}$ and $w \in \mathbb{R}^d, b \in \mathbb{R}$ \begin{equation} \int_{K}\mathbb{1}_{[c_1, \ c_2]}(ax + b)\,d\mu(x)=0 \end{equation} I explained this by using the dominant convergence theorem: \begin{equation*} \lim\limits_{a \rightarrow \infty}{\int_{K}\varrho(ax+b)\,d\mu(x)}=\mu(\{x > 0\})+\varrho(b) \mu(\{x=0\})=0. \end{equation*} If we send $b$ towards negative infinity we get that $\mu(\{x > 0\})=0$. This would imply what we need, but only for parameters $w, b, c_i$ with certain properties (being positive and so on): \begin{equation*} \mu(\{ax + b > c_2\})-\mu(\{ax + b > c_1\})=\int_{K}\mathbb{1}_{[c_1, \ c_2]}(ax + b)\,d\mu(x)=0 \end{equation*} So I am not really sure how they made their conclusion in the proof. If anyone has any ideas, your help is much appreciated. Thank you!

Note: For anyone who is interested in the rest of the proof: You can now extend this integral to step functions and then with the density argument to bounded continuous functions, which include of course the exponential function. Now you have the Fourier transformation of the measure is zero and therefore $\mu=0$.