How to calculate the differential of vector/matrix?

52 Views Asked by At

Suppose we have $L=AXB$, where $A\in R^{m\times n}$, $X\in R^{n\times p}$ and $B\in R^{p\times q}$. Then, how can we obtain the following differential: $$\frac{\partial L}{\partial X}.$$

If possible, could you please give me some hints? Thanks and best regards.

1

There are 1 best solutions below

0
On

The quantity that you are requesting is a 4th order tensor, which cannot be expressed nicely in matrix notation.

So what one typically does is take the vec of both sides of the equation $$ \eqalign { {\rm vec}(L) &= {\rm vec}(AXB) &= (B^T\otimes A)\,{\rm vec}(X) \cr }$$ where $\otimes$ represents the Kronecker product.

This is just a matrix-vector equation, and you should know how to take the derivative of that.

If you really did want the 4th order tensor, then you should probably switch to index notation, e.g. $$ \eqalign{ L_{ij} &= A_{ip}\,X_{ps}\,B_{sj} \cr dL_{ij} &= A_{ip}\,\,dX_{ps}\,\,B_{sj} \cr }$$ Then use the fact that $$ \eqalign{ \frac{\partial X_{ps}}{\partial X_{km}} &= \delta_{pk}\,\delta_{ms} \cr }$$