I'm studying recurrent neural networks which often use tanh as an activator function which is not periodic. However in research and papers it's shown that these recurrent neural nets can exhibit oscillatory, periodic behavior.
If one where to use a linear supperposition of tanh's with weights such as ->
A(n+1) = Tanh( Sum(Tanh for B (n,index)WeightConstant(index), index) )
B(n+1) = Tanh( Sum(Tanh for A (n,index)WeightConstant(index), index) )
I suppose the gut's of the question, is how can one find the conditions for which arbitrary combination of non-periodic functions can create periodic ones? I've doing something with Fourier Expansion and Transformations, but I'm lost.