Scalar-by-matrix derivative involving trace and Hadamard product

Question

I am quite new to matrix calculus, and I am trying to find the scalar-by-matrix derivative to a seemingly simple problem but have yet to find a solution online. I am trying to find \begin{align} \frac{\partial}{\partial \mathbf{X}}\text{Tr}\left((\mathbf{X}\circledast\mathbf{A})^T\mathbf{B}(\mathbf{X}\circledast\mathbf{A})\right), \end{align} where $\circledast$ denotes the Hadamard product. Does anyone know how to compute the solution?

Answer 1

Define a new variable.$$\eqalign{ Y &= A\circledast X \quad\implies\; dY = A\circledast dX \\ }$$ Write the function in terms of this new variable. Then calculate its differential and gradient. $$\eqalign{ \phi &= {\rm Tr}(Y^TBY) \\&= B:YY^T \\ d\phi &= B:(Y\,dY^T+dY\,Y^T) \\ &= (B+B^T):dY\,Y^T \\ &= (BY+B^TY):dY \\ &= (BY+B^TY):A\circledast dX \\ &= A\circledast (BY+B^TY):dX \\ \frac{\partial \phi}{\partial X} &= A\circledast (BY+B^TY) \\ \\}$$ NB: In the above, a colon denotes the trace/Frobenius product, i.e. $\;P:Q = {\rm Tr}(P^TQ)$.
The properties of the trace allow terms in a trace product to be rearranged in any number of ways. $$\eqalign{ P:Q &= P^T:Q^T &= Q:P \\ P:QR &= PR^T:Q &= Q^TP:R \\ }$$ The Hadamard product is commutative, allowing its terms to be rearranged as well. $$\eqalign{ P\circledast Q &= Q\circledast P \\ }$$ Finally, the Hadamard and trace products commute with one another. $$\eqalign{ P:Q\circledast R &= P\circledast Q:R \\ }$$