Suppose $B$ is $m \times n$ and $\Sigma$ is $n \times n$ and symmetric and $B \Sigma B^T$ is also symmetric. I am trying to obtain
the following:
$$\frac{\partial \log( \det[B \Sigma B^T])}{\partial B}$$
and
$$\frac{\partial \log( \det[B \Sigma B^T])}{\partial \Sigma}$$
Now from the matrix cookbook, I know that (1)$$\frac{dlog(det(Y))}{dY} = \operatorname{tr}(Y^{-1}dY) $$.
My attempt is to use chain rule (I am not sure if my chain rule specification is correct) and applying (1):
$$\frac{\partial \log( \det[B \Sigma B^T])}{\partial B} = \frac{\partial \log( \det[B \Sigma B^T])}{\partial B\Sigma B^T} \frac{\partial B\Sigma B^T}{\partial B} = \operatorname{tr}((B \Sigma B^T)^{-1})\frac{\partial B\Sigma B^T}{\partial B} $$
Question 1: Is my chain rule specification correct ?
Question 2: I do not know how to compute $\frac{\partial B \Sigma B^T}{\partial B}$
Similarly, applying the same logic to $\frac{\partial \log( \det[B \Sigma B^T])}{\partial \Sigma}$, which would result in the following
$$\frac{\partial \log( \det[B \Sigma B^T])}{\partial \Sigma} = \frac{\partial \log( \det[B \Sigma B^T])}{\partial B\Sigma B^T} \frac{\partial B\Sigma B^T}{\partial \Sigma} = \operatorname{tr}((B \Sigma B^T)^{-1})\frac{\partial B\Sigma B^T}{\partial \Sigma} = \operatorname{tr}((B \Sigma B^T)^{-1})(B^TB)$$
Question 3: Is $\frac{\partial B\Sigma B^T}{\partial \Sigma} = B^TB$ correct ?
For typing convenience, let $$M=B\Sigma B^T$$ Write the function in terms of this new matrix, and find its differential. $$\eqalign{ f &= \log(\det(M)) \cr\cr df &= d\log(\det(M)) \cr &= d{\rm tr}(\log(M)) \cr &= M^{-T}:dM \cr &= M^{-T}:(dB\,\Sigma B^T+B\,d\Sigma\,B^T+B\Sigma\,dB^T) \cr &= (M^{-T}B\Sigma^T+M^{-1}B\Sigma):dB + (B^TM^{-T}B):d\Sigma \cr }$$The colon denotes the inner/Frobenius product, which is a compact infix notation for the trace $$A:B={\rm tr}(A^TB)$$
Thus the gradients are $$\eqalign{ \frac{\partial f}{\partial B} &= M^{-T}B\Sigma^T+M^{-1}B\Sigma \cr \frac{\partial f}{\partial\Sigma} &= B^TM^{-T}B \cr }$$