I'm working on the field of Complex and Hyper-complex Neural Networks. Therefore I see a lot of papers using both analytics and non-analytics functions without specific justifications. Therefore, I am wondering what is the theorical impact of both method on the behavior of a neural net ? Let's define the sigmoid function of a real : $$ \sigma(x)= \frac{1}{1+e(-x)}$$
It exist first a fully complex and analytic sigmoid activation function to a complex-number : $$z=x+iy$$ defined by : $$ \sigma(z)= \frac{1}{1+e(-z)}$$
And also a non-analytic one : $$ \sigma(z)= \frac{1}{1+e(-x)} + i\frac{1}{1+e(-y)}$$
Can someone explain me what are the intuitions behind the usage of both methods ? Is the analytic one only able to learn analytics functions ?
Thanks you !