Derivative of a trace with Kronecker product

Question

Say I have the following expression: $f=\mathbf{Tr} (S^T L S)$,where $S=(B\bigotimes A)H$ Then what's the derivative of $f$ with respect to $A$ and $B$ ?

Answer 1

First find the SVD of $$H=\sum_k(\sigma_ku_k)v_k^T = \sum_kw_kv_k^T$$ Then use the trace/Frobenius product $\big(A\!:\!B={\rm Tr}(A^TB)\big)$ to rewrite the function before finding its differential and the gradients. $$\eqalign{ f &= L:SS^T \cr\cr df &= L:(dS\,S^T+S\,dS^T) \cr &= (L+L^T):dS\,S^T \cr &= (L+L^T)\,S:dS \cr &= (L+L^T)\,S:(dB\otimes A+B\otimes dA)\,H \cr &= \sum_k\,\,(L+L^T)\,S:(dB\otimes A+B\otimes dA)\,w_kv_k^T \cr &= \sum_k\,\,(L+L^T)\,Sv_k:(dB\otimes A+B\otimes dA)\,w_k \cr &= \sum_k\,\,q_k:{\rm vec}(AW_k\,dB^T+dA\,W_kB^T) \cr &= \sum_k\,\,Q_k:(AW_k\,dB^T+dA\,W_kB^T) \cr &= \sum_k\,\,Q_kBW_k^T:dA + Q_k^TAW_k:dB \cr\cr \frac{\partial f}{\partial A} &= \sum_k\,Q_kBW_k^T,\,\,\,\,\,\,\, \frac{\partial f}{\partial B} = \sum_k\,Q_k^TAW_k \cr }$$ where $$\eqalign{ {\rm vec}(Q_k) &= q_k &= (L+L^T)\,Sv_k \cr {\rm vec}(W_k) &= w_k &= \sigma_k v_k \cr }$$