I'm trying to numerically find the maximum of $$ f(z) = Tr[ A\;U B\; U^H], \quad U=U(z,z^*),\\ A,B,U\in\mathbb{C}^{n\times n},\;\; A=A^H, \quad B=B^H $$ using gradient descent(ascent) w.r.t. $z \in {C}$. As I understand, I have to use gradient scheme $z_{n+1} = z_n + \eta\;2 \frac{\partial f(z,z^*)}{\partial z^*}$, so I have to calculate the derivative $ \frac{\partial f(z,z^*)}{\partial z^*}$. Here is calculation: $$\begin{align} &dTr[ A\; U B\; U^H] \\ &= Tr[ B\; U^H A\; \color{red}{dU } ] + Tr[ B^T U^T A^T \color{blue}{dU^*}] \\ &\left< X:= B\; U^H A \right>\\ &= Tr[ X\;\; (U'_z\;\; \color{red}{dz} + U'_{z^*}\;\;\color{blue}{dz^*}) ] \\ &+ Tr[ X^* ({U^*}'_z \color{red}{dz} + {U^*}'_{z^*} \color{blue}{dz^*})] \\ & \left< \text{using } {f^*}'_z = (f'_{z^*})^*\text{ and } {f^*}'_{z^*} = (f'_z)^* \right> \\ &= Tr[ X \;\;\; (U'_z \quad \color{red}{dz} + U'_{z^*}\quad \color{blue}{dz^*}) ] \\ &+ Tr[ X^* ( (U'_{z^*})^* \color{red}{dz} + (U'_z)^* \color{blue}{dz^*})] \end{align}$$ Denote $U'_z = V,\;\; U'_{z^*} = W$. Then $$\begin{align} dTr[] &= Tr[ X \; V + X^* W^*] \color{red}{dz} \\ &+ Tr[ X \; W + X^* V^*] \color{blue}{dz^*} \end{align}$$ Finally $ \frac{\partial f}{\partial z^*} = Tr[ B\; U^H A\; W] + Tr[B\; U^H A\; V]^* $
Am I correct? I'm pretty unsure here, because I've never studied matrix or Wirtinger calculus.
(U(z) is squeeze operator here, if you are interested:)