Matrix derivative of structured matrix (with constraint)

Question

Let

$$c (A,B) := \log | ABA' + I - \mbox{diag}(ABA')|$$

where matrix $A$ is not necessarily square, matrix $B$ is symmetric, and $I$ is the identity matrix. How to obtain the derivatives $\frac{dc}{dA}$ and $\frac{dc}{dB}$?

Answer 1

Use $(\odot)$ to represent the elementwise/Hadamard product and $(:)$ for the trace/Frobenius product, i.e. $$A:B = {\rm Tr}(A^TB)$$ Define the matrices $$\eqalign{ I &= {\rm identity\,matrix} \cr J &= {\rm matrix\,of\,all\,ones} \cr F &= J-I \cr X &= ABA^T \cr Y &= I + F\odot X \cr Z &= F\odot Y^{-1} =\,\, Y^{-1} - {\rm diag}(Y^{-1}) \cr }$$ NB: $\,$ All of these matrices are symmetric.

Write the function in terms of these new variables. Then find its differential and gradients. $$\eqalign{ c &= \log(\det(Y)) \cr dc &= Y^{-1}:dY \cr &= Y^{-1}:(F\odot dX) \cr &= Z:dX \cr &= Z:(dA\,B\,A^T + A\,B\,dA^T + A\,dB\,A^T) \cr &= (ZAB^T+Z^TAB):dA + (A^TZA):dB \cr &= 2ZAB:dA \,\,+\,\, A^TZA:dB \cr \frac{\partial c}{\partial A} &= 2ZAB,\quad \frac{\partial c}{\partial B} = A^TZA \cr\cr }$$ To follow the above derivation, you need to know a couple of things.

$1)$ The cyclic property of the trace allows the terms in a Frobenius product to be rearranged in a number of different ways, e.g. $$\eqalign{ A:BC = B^TA:C = AC^T:B = etc }$$ $2)$ The Hadamard and Frobenius products are commutative and mutually commutative $$\eqalign{ A:B &= B:A \cr A\odot B &= B\odot A \cr A:(B\odot C) &= (B\odot A):C =\,\, (A\odot B\odot C): J \cr }$$