I am trying to program my own activation function for a neural network.
I am trying to do the same thing as for the Bipolar sigmoid function in this example but for the hyperbolic tangent.
This is how its done for the bipolar sigmoid function: $$ f(x) = {2 \over (1 + e^{-\alpha x} ) - 1} $$
$$ f'(x) = {\alpha (1 - f(x) f(x) )\over 2 } $$
With $y = f(x)$ $$ f'(y) = {\alpha (1 - y y )\over 2 } $$
For the hyperbolic tangent $$ f(x) = tanh(x) $$
$$ f'(x) = sech^2(x) $$
$$ f'(y) = ?? $$
How can I find f '(y) ?
If you mean what I think you mean (turning a derivative into a differential equation basically):
$$f'(x)=sech^2 x = 1 - tanh^2 x = 1- f(x)^2 $$