Is it possible to explain in layman's terms how the sigmoid formula ($y=\frac{1}{1+e^{-x}}$) defines some aspects of the function? If it is, could someone please give a layman's explanation?
For example, with the linear function ($y=\beta_0+\beta_1x$) I understand it in the following way: Starting from the intercept ($\beta_0$), $\beta_1$ denotes the number of units that $y$ changes when $x$ changes by 1 unit, and from that it is then immediately clear how this creates a straight slope.
Please note, I'm not asking for an intuition regarding the properties of the sigmoid function, (e.g. centered at 0.5, bounded between 0 and 1) but rather an explanation of how the formula defines aspects of the sigmoid function.
For example, some aspects which one may be able to explain in layman's terms are:
1) What bounds it between 0 and 1?
2) What centers it around 0.5?
3) What gives it an S shape?
I'm also aware that it could be one of those things that I'll just have to accept as being "it just is the way it is..." However, it would be more satisfying to "get" the actual formula behind it.
Thanks!
I'll just address the properties you mention. Denote the function by $$S(x) = \frac{1}{1 + e^{-x}} .$$
Hints
Use the fact that $1 < 1 + e^{-x}$ and take reciprocals.
This is easier to see if we rewrite $S(x)$ more symmetrically as $$S(x) = \frac{1}{2} + \frac{1}{2} \cdot \frac{e^{x / 2} - e^{-x / 2}}{e^{x / 2} + e^{-x / 2}} = \frac{1}{2} + \frac{1}{2} \tanh \frac{x}{2}$$ The second term in either of the latter two expressions is odd in $x$. (Remark: If you're familiar with the properties of $\tanh$, this expression makes some properties of $S$ much clearer.)
This depends on what you mean by "$S$-shaped", but if you mean that its graph is symmetric about a point $(x_0, y_0)$ (which we concluded in (2)) and concave down on $(x_0, \infty)$ (and so by symmetry concave up on $(-\infty, x_0)$), it's enough to compute the second derivative, $S''(x)$.