What is the form of the gradient with respect to a complex matrix?

49 Views Asked by At

We know that the definition of the gradient of a real-valued function with respect to a parameter is that the steepest incremental direction at the point of the current parameter position. The form of the gradient always become more complicated when the parameter is "complex."

What about the gradient of a real-valued function $f(\mathbf{X})$ with respect to complex matrix $\mathbf{X}\in\mathbb{C}^{M\times N}$?

Considering some toy examples, I think the gradient of $f(\mathbf{X})$ with respect to $\mathbf{X}$ should be $\nabla_{\mathbf{X}^*} f(\mathbf{X})\in\mathbb{C}^{M\times N}$ where $$\nabla_{\mathbf{X}^*}=\begin{bmatrix} \frac{\partial}{\partial x_{1,1}^*} & \frac{\partial}{\partial x_{1,2}^*} & \dots & \frac{\partial}{\partial x_{1,N}^*}\\ \frac{\partial}{\partial x_{2,1}^*} & \frac{\partial}{\partial x_{2,2}^*} & \dots & \frac{\partial}{\partial x_{2,N}^*}\\ \vdots & & \ddots & \vdots\\ \frac{\partial}{\partial x_{M,1}^*} & & \dots & \frac{\partial}{\partial x_{M,N}^*} \end{bmatrix}$$ $\mathbf{X}^*\in\mathbb{C}^{M\times N}$ denotes a complex conjugate.

Compared to this matrix case, a vector cases are already mentioned in a few articles. ex) Wirtinger Calculus see equation (A.3.6) link: https://onlinelibrary.wiley.com/doi/pdf/10.1002/0471439002.app1