Complex valued neural nets (CVNNs) use complex number natively and Wirtinger calculus for computing gradients, as I've read. I have trouble finding an explicit explanation of the clear difference between having a "standard" real valued neural net (RVNN) with 2 channels (real and imaginary) to implement complex numbers, e.g., complex learnable weights. One intuition seems to be that CVNNs manage these complex weight together instead of independently for real and imaginary channels. Although i guess that this is a matter of implementation of these CVNNs... are we sure/how to really enforce that the complex number is treated as a single entity in CVNNs (and not internally treated as 2 real numbers i.e. real and imag or phase and magnitude)?
Also i guess that CVNNs could handle complex numbers at all layers i.e. not just a 2 channel real imag input and then real convolutions with n layers which do not explicitly deal with complex numbers... but this is not clear... can someone clarify?
Some papers also mention something different w.r.t. complex multiplication which - i know is a rotation and scaling - but again not clear what is the impact ...
If complex weights are really treated jointly then the number of parameters of the network could be divided by 2, thus improving statistical efficiency, but again , maybe it stems from the Wirtinger calculus which i do not understand enough for sure (I just have some notions that it generalizes complex derivatives but i do not see how it can solve the problem of jointly treating the real & imag. parts of complex numbers instead of computing partial derivatives w.r.t real and imaginary parts independently?