I have the following function of a trace $$ f(\mathbf{x})=\mathrm{tr}(AM(\mathbf{x})A^T) $$
with $A$ and $M$ two $n\times n$ square matrices, and $\mathbf{x}$ a vector in $\mathbb{R}^m$. The matrix $M$ is a known explicit function of $\mathbf{x}$, and its derivative $M'=\frac{\mathrm{d}M}{\mathrm{d}\mathbf{x}}$, that is a $(n\times n \times m)$ tensor, is known.
I am really struggling to find an explicit elegant form for $f'=\frac{\mathrm{d}f}{\mathrm{d}\mathbf{x}}$. Do you have some hint? Thanks
$ \def\l{{\large\ell}} \def\G{{\cal G}} \def\LR#1{\left(#1\right)} \def\op#1{\operatorname{#1}} \def\trace#1{\op{Tr}\LR{#1}} \def\qiq{\quad\implies\quad} \def\p{\partial} \def\grad#1#2{\frac{\p #1}{\p #2}} \def\c#1{{#1}} \def\gradLR#1#2{\LR{\grad{#1}{#2}}} \def\Sij{\sum_{i=1}^n\sum_{j=1}^n} \def\Sk{\sum_{k=1}^n} \def\Sl{\sum_{{\l}=1}^{n^2}} $For typing convenience, define the variables $$\eqalign{ B &= A^TA \\ \G &= \grad{M}{x} \qiq \G_{ijk} &= \grad{M_{ij}}{x_{k}} \\ }$$ and the single and double dot products $$\eqalign{ \G\cdot x &= \Sk\,\G_{ijk}\,x_{k} \\ B:\G &= \Sij\,B_{ij}\,\G_{ijk} \\ }$$ Use the above notation to write the objective function, then calculate its differential and gradient. $$\eqalign{ f &= B:M \\ df &= B:dM \;=\; B:\G\cdot dx \\ \grad{f}{x} &= B:\G \\\\ }$$
To avoid double-dot products, replace the $(i,j)$ indexes with a single long index $$\l = i+\LR{j-1}n$$ This has the effect of flattening $B$ into a vector $b$ and $\G$ into a matrix $G,\,$ i.e. $$\eqalign{ \grad{f}{x_k} &= \Sl b_{\l}\,G_{\l k} \qiq \grad{f}{x} = b\cdot G \\ }$$