How can we compute the derivative of the following quantity with respect to $\bf{\Sigma}$?
$$ \phi = {\bf{x}}^\top({\bf \Sigma^{-1}\otimes I){\bf x}} $$
Edit: I have tried to solve this problem by referring to this.
$$ \phi = \text{tr}({\bf xx}^\top({\bf \Sigma}^{-1} \otimes {\bf I})). $$
Now I need to find a factorization of ${\bf xx}^\top = {\bf A} \otimes {\bf B}$. I can factorize it as ${\bf x}^\top \otimes {\bf x}$, but this doesn't agree with the dimension of ${\bf \Sigma}$ and the identity matrix to simplify the problem further. So I am not sure how to proceed this.
$ \def\l{\left} \def\R#1{{\mathbb R}^{#1}} \def\r{\right} \def\lr#1{\l(#1\r)} \def\c#1{\color{red}{#1}} \def\R#1{{\mathbb R}^{#1}} \def\grad#1#2{\frac{\partial #1}{\partial #2}} $For typing convenience, define the matrix variables $$\eqalign{ S &= \Sigma &\in\R{n\times n} \\ X &= xx^T \quad&\in\R{np\times np} }$$ Let's also use a colon as a convenient product notation for the trace, i.e. $$\eqalign{ A:B &= \sum_{i=1}^m\sum_{j=1}^n A_{ij}B_{ij} \;=\; {\rm Tr}(AB^T) \\ A:A &= \big\|A\big\|^2_F\\ }$$ Write the function using the above notation $$\phi = X:\lr{S^{-1}\otimes I_p}$$ Let $\{e_k\}$ denote the $k^{th}$ column of $I_p$ and define their block-matrix analogs $$\eqalign{ E_k &= (I_n\otimes e_k) \;\in\; \R{pn\times n} \\ }$$ Note that $$\eqalign{ \sum_{k=1}^p E_kE_k^T &= \sum_{k=1}^p \lr{I_n\otimes e_k}\lr{I_n\otimes e_k}^T \\ &= (I_n\,I_n^T)\otimes\lr{\sum_{k=1}^p e_ke_k^T} \\ &= I_n\otimes I_p \\&= I_{np} \\ }$$ Now expand the cost function $$\eqalign{ \phi &= \lr{I_{np}\,X\,I_{np}}:\lr{S^{-1}\otimes I_p} \\ &= \sum_{i=1}^p\sum_{k=1}^p \lr{E_iE_i^T}X\lr{E_kE_k^T}:\lr{S^{-1}\otimes I_p} \\ &= \sum_{i=1}^p\sum_{k=1}^p \lr{E_i^T\,X\,E_k}:E_i^T\lr{S^{-1}\otimes I_p}E_k \\ &= \sum_{i=1}^p\sum_{k=1}^p \lr{E_i^T\,X\,E_k}:\lr{I_n\otimes e_i^T}\lr{S^{-1}\otimes I_p}\lr{I_n\otimes e_k} \\ &= \sum_{i=1}^p\sum_{k=1}^p \lr{E_i^T\,X\,E_k}:S^{-1}\otimes\lr{e_i^Te_k} \\ &= \sum_{i=1}^p\sum_{k=1}^p \lr{E_i^T\,X\,E_k}:S^{-1}\delta_{ik} \\ &= \sum_{k=1}^p \lr{E_k^T\,X\,E_k}:S^{-1} \\ &\equiv Y:S^{-1} \\ }$$ Now that the independent variable has been isolated on the RHS, the rest of the derivation is a breeze. $$\eqalign{ \phi &= Y:S^{-1} \\ d\phi &= Y:dS^{-1} \\ &= Y:\lr{-S^{-1}\,dS\,S^{-1}} \\ &= -S^{-T}YS^{-T}:dS \\ \grad{\phi}{S} &= -S^{-T}YS^{-T} \\ }$$ If $S$ is symmetric, then this can be simplified a bit $$\eqalign{ \grad{\phi}{S} &= -S^{-1}YS^{-1}\quad\quad\quad \\\\ }$$
NB: $\;$ The properties of the underlying trace function allow us to rearrange the terms in a colon product in a number of different ways, e.g. $$\eqalign{ A:B &= A^T:B^T \\ A:BC &= B^TA:C &= AC^T:B \\ A:B &= B:A \\ }$$