I would like to code this neural net activation function, using the C language:
$$f(x) = 1.7159 \tanh( \frac{2}{3} x )$$
and I will also need to code its derivative.
I've read that the derivative of $\tanh(x)$ is $\operatorname{sech}^2(x)$, but since C doesn't have a hyperbolic secant function I will need to use $\cosh$, i.e. the derivative of $\tanh(x)$ is $1\over \cosh^2(x)$, I think.
Since my knowledge of calculus is very rusty, my best attempt for the derivative of the above function is:
$$1 \over \cosh^2(\frac{2}{3}x)$$
Is this correct?
The derivative is $\frac{2*1.7159}{3} / \cosh^2 \frac{2x}{3}$ by chain rule. Note that it can be also written as $\frac{2*1.7159}{3} (1- \tanh^2 \frac{2x}{3})$ which might make slightly more sense as you only need to find $\tanh$ then (if you want to find $f(x)$ and $f'(x)$ at the same time). All the useful relations can be found in wikipedia.