I was reading an article about image processing and I came across sigmoidal activation function and tanh like in this article:
But I'm struggling to understand the concept behind the nonlinearity here and what has to do with tanh, could anyone clarify this idea, please
$\psi(t)=\mathrm{tanh}(\alpha t)$ is a nonlinear transformation.
For $|\alpha t|$ close to $0$, the function is actually an approximately linear function of $t$, with $\psi(t) \approx \alpha t$.
By the time $|\alpha t| \ge 2$, the function's nonlinearity is very apparent, almost not really depending on $\alpha t$ at all, but just the sign of $\alpha t$, so $\psi(t) \approx \mathrm{sgn}(\alpha t)$.
In summary, $\psi(t)=\mathrm{tanh}(\alpha t)$ is a nonlinear tranformation with a "softer" transition than the basic $\psi(t)=\mathrm{sgn}(t)$ nonlinear transformation.