partial derivatives of matrix

Question

I have trouble in calculating partial derivatives of matrix recently. The problem is described as following \begin{align} \frac{\partial \log \det\left({\mathbf{I}+\mathbf{\Sigma Q}}\right)}{\partial \mathbf{Q}} \end{align} where $\mathbf{I}$ is identity matrix. $\mathbf{\Sigma}$ and $\mathbf{Q}$ are square matrix. $\det(\cdot)$ is determinant operation. I wanna compute this formula. thanks a lot.

Answer 1

Solution: \begin{align} \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial \mathbf{Q}}= \left[\begin{array}{ccc} \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{11}}&\cdots &\frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{1,u}}\\ \vdots & \ddots &\vdots\\ \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{u,1}} &\cdots & \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{u,u}} \end{array} \right] \end{align} So the key is to find $\frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{i,j}}$ for $(i,j)\in [u,u]$.

To this end,we apply the fact \begin{align} \frac{\partial \log \det (\mathbf{A})}{\partial x}=\text{tr}\left({\mathbf{A}^{-1}\frac{\partial \mathbf{A}}{\partial x}}\right) \end{align} where $\mathbf{A}$ is function of $x$, i.e., $\mathbf{A}=\mathbf{A}(x)$. (The proof of above formula can be given by $\frac{\partial \log \det(A)}{\partial x}=\frac{1}{\det(\mathbf{A})}\frac{\partial \det(\mathbf{A})}{\partial x}$ and Jacobi's formula).

With this knowledge, we have \begin{align} \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{i,j}} &\overset{(a)}{=}\text{tr} \left({\left({\mathbf{I}+\mathbf{\Sigma Q}}\right)^{-1}\mathbf{\Sigma}\boldsymbol{e}_i\boldsymbol{e}_j^T}\right)\\ &\overset{(b)}{=}\boldsymbol{e}_j^T \left({\left({\mathbf{I}+\mathbf{\Sigma Q}}\right)^{-1}\mathbf{\Sigma}}\right) \boldsymbol{e}_i \end{align} where $\boldsymbol{e}_i$ is a vector with all zero elements expect i-th element being 1 in step $(a)$, and $(b)$ holds via the property of trace operation.

Here, $\mathbf{\Sigma}$ and $\mathbf{Q}$ are symmetric matrix. We then have \begin{align} \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{i,j}} =\boldsymbol{e}_i^T \left({\left({\mathbf{I}+\mathbf{\Sigma Q}}\right)^{-1}\mathbf{\Sigma}}\right) \boldsymbol{e}_j \end{align} Thus, we can get \begin{align} \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial \mathbf{Q}}=(\mathbf{I}+\mathbf{\Sigma Q})^{-1}\mathbf{\Sigma} \end{align}

By the way, if $\mathbf{\Sigma}$ and $\mathbf{Q}$ are not symmetric, then the result is given by \begin{align} \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial \mathbf{Q}}=\mathbf{\Sigma}^T((\mathbf{I}+\mathbf{\Sigma Q})^{-1})^T \end{align}

Answer 2

Define the matrix $$X=(I+\Sigma Q)$$ Write the function in terms of this new matrix, then find the differential and gradient. $$\eqalign{ \lambda &= \log\det X \cr d\lambda &= X^{-T}:dX = X^{-T}:\Sigma\,dQ = \Sigma^TX^{-T}:dQ \cr \frac{\partial\lambda}{\partial Q} &= \Sigma^TX^{-T} \cr }$$ Since $(\Sigma, Q)$ are symmetric, the gradient can be simplified to $$\frac{\partial\lambda}{\partial Q} = \Sigma\,(I+Q\Sigma)^{-1}$$