Scaling function defined over infinite range down to finite

379 Views Asked by At

Suppose you have a simple sigmoidal function: $$f(x)=\frac{1}{1+e^{-x}}$$

The function itself isn't very particular:

Sigmoidal function

But it's first derivative is:

$$f'(x)=\frac{e^{-x}}{(1+e^{-x})^2}$$

First order derivative over sigmoidal function

The function has a lot of useful properties which can be used in variety of engineering tasks, integral over first order derivative is equal to 1:

$$\int_{-\infty}^\infty \frac{e^{-x}}{(1+e^{-x})^2}\ dx = 1$$

It's shape is close to normal distribution and can be used in variety of signal processing tasks. And multiplying argument over $2\ π$ we get nice distribution of values in $-1..1$ range:

Sigmoidal over 2 π x

There is one problem however, this function is defined over infinite range and therefore should be adapted for set of cases where range is required to be finite.

One way to do it is to wrap function argument $x$ into another function that is defined over limited range and approaches towards infinity like tangent function:

Tangent

Which yields to:

$$f'(\tan(x))=\frac{e^{-\tan(x)}}{(1+e^{-\tan(x)})^2}$$

Although this approach allows to see the behavior of function when argument approaches infinity but it is also significantly affects the shape of function significantly plus it becomes impossible to compute the integral analytically:

Sigmoid over tan over x

So from this standpoint trigonometry doesn't play really nice with exponential functions, so maybe let's try to replace tangent with inverse hyperbolic tangent which closely resembles the behavior of tangent:

$$arctanh(x)=\frac{1}{2}(\ln(1+x)-\ln(1-x))$$

atanh(x)

$$f'(arctanh(x))=\frac{e^{-arctanh(x)}}{(1+e^{-arctanh(x)})^2}$$

And the picture is even worser:

f over atanh over x

So this is the place where I have no more ideas. I would like to shrink the range of derivative of sigmoidal function into finite range, is there a way to do it in a way that preserves the shape of the original function, and potentially allows for analytical computation of an integral?