Given $F,f:\mathbb R\to\mathbb R$ such that $F'=f$ and $\pmb a,\pmb b\in\mathbb R^n$, compute $$\frac{d}{d\pmb X}\left(\pmb a^T F\left(\pmb X\right)\pmb b\right)$$ where $\pmb X\in\mathbb R^{n\times n}$.
My guess is $\operatorname{diag}(\pmb b) f(\pmb X)^T\operatorname{diag}(\pmb a)$ but I would like to
(i) confirm it and
(ii) see if there's a better way than using indices.
First, note that $a^TF(X)b=\operatorname{tr}(a^TF(X)b)=\operatorname{tr}(ba^TF(X))$. So using differentials we have
$$d\operatorname{tr}(ba^TF(X))=\operatorname{tr}(ba^TdF(X))=\operatorname{tr}(ba^Tf(X)\odot dX)=ab^T\cdot f(X)\odot dX=$$ $$=ab^T\odot f(X)\cdot dX$$ As $\operatorname{tr}(A^TB)=A\cdot B$, where $A\cdot B=\sum_{i,j}A_{i,j}B_{i,j}$ for matrices of the same dimensions. $\odot$ - Hadamard product.
From this we see that $$\frac{d}{dX}\left(a^T F\left(X\right)b\right)=ab^T\odot f(X)$$ Also note $ab^T\odot f(X)=\operatorname{diag}(a)f(X)\operatorname{diag}(b)$.
I have used notations and properties of operations from this paper.