Can somebody tell me how to arrive at the following expressions
\begin{align} \mathbf{D} &= (\mathbf{C}^{-1}\mathbf{t}\mathbf{t}^{T}\mathbf{C}^{-1}-\mathbf{C}^{-1}) \mathbf{\Phi} \mathbf{A}^{-1} \\ &= \beta[(\mathbf{t}-\mathbf{y})\boldsymbol{\mu}^{T} - \mathbf{\Phi}\Sigma] \end{align}
where \begin{equation} \mathbf{C} = \beta^{-1}\mathbf{I} + \mathbf{\Phi} \mathbf{A}^{-1} \mathbf{\Phi}^{T}, \\ \boldsymbol{\mu} = \beta^{-1}\mathbf{\Sigma}\mathbf{\Phi}^{T}\mathbf{t}, \\ \boldsymbol{\Sigma} = \left( \mathbf{A} + \beta \mathbf{\Phi}^{T} \mathbf{\Phi} \right)^{-1} \end{equation}
which contain the derivatives of the following function \begin{equation} \mathcal{L} = -\frac{1}{2}\big[N\log 2\pi + \log\left|\mathbf{C}\right| + \mathbf{t}^{T}\mathbf{C}^{-1}\mathbf{t} \big] \end{equation}
with respect to the elements $\phi_{mn}$ of the matrix $\mathbf{\Phi}$ (i.e. $\mathbf{D}_{mn} = \partial \mathcal{L} / \partial \phi_{mn}$)?
Note that bold small letters e.g $\mathbf{t}$ denote $N\times 1$ vectors, bold capital letters $N\times N$ matrices, and $\mathbf{A}$ is diagonal.
Also, the author of the paper where I found it makes frequent use of the following identities (by the Woodbury inversion lemma) which might be helpful:
\begin{align} \mathbf{C}^{-1} = (\beta^{-1}\mathbf{I} + \mathbf{\Phi} \mathbf{A}^{-1} \mathbf{\Phi}^{T} )^{-1} = \beta \mathbf{I} - \beta \mathbf{\Phi}\left( \mathbf{A} + \beta \mathbf{\Phi}^{T} \mathbf{\Phi} \right)^{-1} \mathbf{\Phi}^{T} \beta = \beta \mathbf{I} - \beta \mathbf{\Phi} \mathbf{\Sigma} \mathbf{\Phi}^{T} \beta \end{align}
Before we begin, note that $\bf C$ is symmetric, since $\bf A$ is diagonal and therefore symmetric.
Also note that the term $\,\,{\bf {\rm log}({\rm det}({\bf C}))}\,\,$ can be replaced by $\,\,{\bf {\rm tr}({\rm log}({\bf C}))}\,\,$ which is easier to work with. And the final term has been written in terms of the Frobenius product, for the same reason.
So start with the expression $$ -2\, L = N\,{\rm log}(2\pi) + {\rm tr}({\rm log}({\bf C})) + {\bf C}^{-1}:{\bf tt}^T $$ and take the differential of both sides $$\eqalign{ -2\,dL &= {\bf C}^{-1}:d{\bf C} - {\bf tt}^T:({\bf C}^{-1}\,d{\bf C}\,{\bf C}^{-1}) \cr &= {\bf C}^{-1}:d{\bf C} - {\bf C}^{-1}{\bf tt}^T{\bf C}^{-1}:d{\bf C} \cr &= ({\bf C}^{-1} - {\bf C}^{-1}{\bf tt}^T{\bf C}^{-1}):d{\bf C} \cr }$$ Now find an expression for $d{\bf C}$ in terms of $d {\bf \Phi}$. $$\eqalign{ {\bf C} &= \beta^{-1}{\bf I} + {\bf \Phi} {\bf A}^{-1}{\bf \Phi}^T \cr d{\bf C} &= d\Phi {\bf A}^{-1}{\bf \Phi}^T + {\bf \Phi} {\bf A}^{-1}d{\bf \Phi}^T \cr &= 2\,\,{\rm sym}(d{\bf \Phi} {\bf A}^{-1}{\bf \Phi}^T) \cr }$$ Substituting $$\eqalign{ -2\,dL &= ({\bf C}^{-1}-{\bf C}^{-1}{\bf tt}^T{\bf C}^{-1}):2\,{\rm sym}(d{\bf \Phi} {\bf A}^{-1}{\bf \Phi}^T) \cr &= 2\,{\rm sym}({\bf C}^{-1} - {\bf C}^{-1}{\bf tt}^T{\bf C}^{-1}):d{\bf \Phi} {\bf A}^{-1}{\bf \Phi}^T \cr &= 2\,({\bf C}^{-1} - {\bf C}^{-1}{\bf tt}^T{\bf C}^{-1}):d{\bf \Phi} {\bf A}^{-1}{\bf \Phi}^T \cr &= 2\,({\bf C}^{-1} - {\bf C}^{-1}{\bf tt}^T{\bf C}^{-1}){\bf \Phi} {\bf A}^{-1}:d{\bf \Phi} \cr\cr dL &= ({\bf C}^{-1}{\bf tt}^T{\bf C}^{-1}-{\bf C}^{-1}){\bf \Phi} {\bf A}^{-1}:d{\bf \Phi} \cr\cr }$$ So the derivative is $$\eqalign{ {\bf D} &= \frac{\partial L}{\partial {\bf \Phi}} &= ({\bf C}^{-1}{\bf tt}^T{\bf C}^{-1}-{\bf C}^{-1}){\bf \Phi} {\bf A}^{-1} \cr }$$ which you can reformulate in terms of {$y,\mu,\Sigma$, etc} if you wish.