Given
- $f:\mathbb{R}^{n\times n}\rightarrow\mathbb{R}$
- $H\in\mathbb{R}^{n\times n}$ is a variable
- $P\in\mathbb{R}^{n\times n}$ is a constant
- $f':\mathbb{R}^{n\times n}\rightarrow\mathbb{R}^{n\times n}$ is the gradient of $f$ i.e. $f_{ij}'(H)=\frac{\delta}{\delta h_{ij}}f(H)$
Is there a matrix expression for $\frac{d}{dH}f(P^{-1}HP)$ ?
Is it perhaps $P^{-1}f'(P^{-1}HP)P$?
Suppose you have calculated the gradient $(G)$ of a function in terms of the variable $Y$ $$ G = \frac{\partial f}{\partial Y} \quad\implies\quad df = G:dY $$ where the colon denotes the trace product, i.e. $\;M:N = {\rm Tr}(M^TN)$
You are then told that $Y$ is not independent, but is actually a function of another matrix $X,\,$ e.g. $$Y=AXB$$ Calculate the gradient $(J)$ with respect to this new variable, by expanding the differential and then performing a change of variables. $$\eqalign{ df &= G:dY \\ &= G:A\,\,dX\,B \\ &= A^TGB^T:dX \\ &= J:dX \\ J &= A^TGB^T \;=\; \frac{\partial f}{\partial X} \\ }$$ In terms of components, this result can be written as $$\eqalign{ \frac{\partial f}{\partial X_{il}} &= \sum_j\sum_k A^T_{ij}\left(\frac{\partial f}{\partial Y_{jk}}\right)B^T_{kl} \\ }$$