Fix $a,b\in\mathbb{R}^d$ and define the function $$ \begin{aligned} f:\operatorname{Mat}_{d\times d} & \rightarrow \mathbb{R}^d\\ A & \mapsto \frac1{a^TA^{-1}b} \cdot A^{-1} b, \end{aligned} $$ where $\cdot$ is scalar multiplication.
I want to differentiate $f$ with respect to $A$. Here's what I have computed so far: $$\begin{aligned} \frac{\partial a^TAb }{\partial A} = & a^Tb\\ \frac{\partial A^{-1}b}{\partial A} = \frac{\partial A^{-1}}{\partial A}b \end{aligned} $$ So.. applying the chain rule a couple times yields $$ \frac{\partial \frac1{a^TA^{-1}b}}{\partial A} = \frac{\partial A^{-1}}{\partial A} \left( \frac{a^Tb}{a^T A^{-1} b} \right). $$
At this point I get stuck... what is $\frac{\partial A^{-1}}{\partial A}$ and what is the scalar-multiplication analogue of the product rule (so I can combine $\frac{\partial \frac1{a^TA^{-1}b}}{\partial A}$ with $\frac{\partial A^{-1}b}{\partial A}$....)
Update: For $A^{-1}$'s derivative I compute: $$ \begin{aligned} 0=&\frac{\partial I}{\partial A}=\frac{\partial (AA^{-1})}{\partial A}=\frac{\partial A}{\partial A}A^{-1} + A\frac{\partial (A^{-1})}{\partial A}\\ \therefore \frac{\partial A^{-1}}{\partial A}=&-A^{-1}IA^{-1} = -(A^{-1})^2= -A^{-2}. \end{aligned} $$
Thanks in advance.
Change the name of the vector $a$ to $c$, and re-purpose the name to vectorize the matrix $$a = \operatorname{vec}\left(A\right)$$ Define the scalar variable $$\eqalign{ \phi &= c^TA^{-1}b \\ d\phi &= -c^TA^{-1}\,dA\,A^{-1}b \\ }$$ Write the function in terms of this scalar.
Then calculate its differential and gradient. $$\eqalign{ f &= \phi^{-1}A^{-1}b \\ df &= d\phi^{-1}A^{-1}b\, + \phi^{-1}dA^{-1}b \\ &= -\phi^{-2}A^{-1}b\,d\phi - \phi^{-1}A^{-1}\,dA\,A^{-1}b \\ &= f\,\phi^{-1}(c^TA^{-1}\,dA\,A^{-1}b) - A^{-1}\,dA\,f \\ &= f\left(c^TA^{-1}\,dA\,f\right) - A^{-1}\,dA\,f \\ &= \left(fc^T-I\right)\left(A^{-1}\,dA\,f\right) \\ &= \left(fc^T-I\right)\left(f^T\otimes A^{-1}\right)da \\ \frac{\partial f}{\partial a} &= \left(fc^T-I\right)\left(f^T\otimes A^{-1}\right) \\ }$$ The requested gradient (of a vector with respect to a matrix) is actually a third-order tensor and cannot be expressed in standard matrix notation. So this vectorized result is about the best you can do.