I haven't found a simple explanation of this anywhere.
I have the following matrix product,
to give $C$,
I need to compute $\frac{dC}{dB}$. I handcomputed the jacobian and it looks like this,
How can I compute the above jacobian in a tensorized way? Is there a nice series of transposes, or matrix multiplications that can result in the above matrix quickly?
Vectorize the matrix equation, then find its differential and gradient. $$\eqalign{ C &= AB = ABI \cr {\rm vec}(C) &= (I^T\otimes A)\,{\rm vec}(B) \cr {\rm vec}(dC) &= (I\otimes A)\,{\rm vec}(dB) \cr \frac{\partial\,{\rm vec}(C)}{\partial\,{\rm vec}(B)} &= (I\otimes A) \cr }$$ Applying the same process to a slightly different equation yields $$\eqalign{ F &= BA^T = IBA^T \cr {\rm vec}(F) &= (A\otimes I)\,{\rm vec}(B) \cr {\rm vec}(dF) &= (A\otimes I)\,{\rm vec}(dB) \cr \frac{\partial\,{\rm vec}(F)}{\partial\,{\rm vec}(B)} &= (A\otimes I) \cr }$$ which is the gradient that you calculated.