Consider a function $\mathbb{C} \rightarrow \mathbb{R}$, $f(z) = |z|^2$. What is the first order derivative of $f(z)$? And what is the gradient of $f(z)$?
I found in the The Matrix Cookbook, it said that the gradient of $f(z)$ is $2z$. Is there any difference between gradient and first order derivative?
Hint: $f = g\circ h$ with $$g:\Bbb C\times\Bbb C\longrightarrow\Bbb C,\qquad(w,z)\longmapsto w\bar z$$ real bilinear and $$h:\Bbb C\longrightarrow\Bbb C\times\Bbb C\qquad z\longmapsto (z,z)$$ real (and complex) linear.