This article discuss how to implement gradient descent on a real-valued cost function. But I"m wondering if the same can be done for a complex valued cost function?
My thinking is it is not possible because you cannot take partial derivatives of a complex valued cost function, whereas you can with a real-valued cost function, and the gradient descent algorithm takes partial derivatives in order to determine what the next step is. Is this correct?
Costs are normally assumed ordered, and there can be no (consistent) order between complex values. Quick: Which is larger, $2 + 3 i$ or $-3 + 2 i$? You could compare them by comparing absolute values, but $\lvert 2 + 3 i \rvert = \lvert -3 + 2 i \rvert$. And you'd be back to real costs that way.