What is the sigmoid *squashing* function?

Question

I've just read the following

The basic unit ("neuron" i) performs the following computation to update its state $y_i$: it computes a weighted sum $v_i$ of its inputs $x:j$ which is passed through a sigmoid squashing function $g ( \cdot )$.

Source: Design of a neural network character recognizer for a touch terminal

I know what a sigmoid function is, but what is a sigmoid squashing function? I have also seen this in the PyBrain documentation.

Answer 1

Fix a reference measure $\mu$.
A sigmoid function $\sigma_0$ is a essentially-bounded map from $\mathbb{R}$ to itself. I.e.: $\sigma_0 \in L^{\infty}_{\mu}(\mathbb{R})$.

While a squashing function $\sigma$ is a sigmoid function satisfying $$ \operatorname{ess-sup}|\sigma| \leq 1 , $$ thus, $\sigma \in Ball_{L^{\infty}_{\mu}(\mathbb{R})}$.

Practical Lingo:

In practical terms, if $\mu$ is the Lebesgue measure, then $\sigma_0:\mathbb{R}\to [m,M]$ for some $m\leq M$, real numbers, while $\sigma:\mathbb{R} \to [-1,1]$.

I think this paper has a rigerous definitio (under the additional assumption of continuity). While this paper hints to the non-continuous version.

Answer 2

The sigmoid squashing function is the same as the sigmoid function.

The term sigmoid squashing function is favored in the neural net community. The logistic function is the classical squashing function. Other sigmoid functions include: arctangent, the hyperbolic tangent, the Gudermannian function, and the error function.

It is called the squashing function because it is tightly bounded to the pair of horizontal asymptotes. Thus, useful in compressing, or squashing, outputs.