I'm new to matrix derivatives, and I'm having a bit of trouble with this one in particular. I have this equation for the function:
$f(x) = M(g(x) ∘ g(x))$
Where M is a non-square matrix, '$∘$' is Hadamard product, and '$g(x)$' returns a matrix.
How would you go about deriving this?
For consistency, I'll use uppercase letters for matrices and lowercase for vectors. I'll also use $$\eqalign{ &A\circ B \cr &A:B = {\rm tr}(A^TB) \cr &A\otimes B \cr }$$ to denote the elementwise/Hadamard, trace/Frobenius, and Kronecker products, respectively.
Then the function of interest is $$F = M(G\circ G)$$ Use the vectorization and diagonalization operators to create some new variables $$\eqalign{ f &= {\rm vec}(F)\cr g &= {\rm vec}(G)\cr \Gamma &= {\rm Diag}(g)\cr A &= I\otimes M \cr J &= \frac{\partial g}{\partial x} &\implies dg = J\,dx \cr }$$ Now we are ready to find the differential and gradient of your function $$\eqalign{ f &= {\rm vec}(M(G\circ G)I) = A\,(g\circ g) \cr df&= 2A\,(g\circ dg) = 2A\Gamma J\,dx \cr \frac{\partial f}{\partial x} &= 2A\Gamma J \cr \cr \frac{\partial {\rm vec}(F)}{\partial x} &= 2\,(I\otimes M)\,\,{\rm Diag}\Big({\rm vec}(G)\Big)\,\,\frac{\partial {\rm vec}(G)}{\partial x} \cr }$$