I am trying to implement an echo state recurrent neural network based on a few papers and I am having trouble replicating their training data used in an example. Rather than type it all out, I've included an image of the training description, the paper's training data output.
Training Output, First 200 Iterations
Unfortunately, I have been unable to get mine to match the example in the paper. My output consistently saturates to 1.0 by the 100th iteration.
What I don't understand is how the output can oscillate when all the constants, inputs, and output of tanh() are all positive. Eventually it HAS to saturate right? I just want to understand what I am missing in how the equation is implemented.