According to this paper: Rectified Linear Units Improve Restricted Boltzmann Machines,
$\sum_{i=1}^N \sigma(x-i+\tfrac{1}{2}) \approx \log(1+e^x)$ (equation 7)
where $\sigma(z) = \frac{1}{1+e^{-x}}$
Would anyone please explain me why it is so?
According to this paper: Rectified Linear Units Improve Restricted Boltzmann Machines,
$\sum_{i=1}^N \sigma(x-i+\tfrac{1}{2}) \approx \log(1+e^x)$ (equation 7)
where $\sigma(z) = \frac{1}{1+e^{-x}}$
Would anyone please explain me why it is so?
The approximation was probably obtained thanks to two integrals encircling the sum (supposed with $N$ large enough), as shown in attachment :
In addition, the figure below shows the relative deviation as a function of $x$