Let's say we have
$${\sigma(\exp(a\cdot X^{-1} \cdot a^\mathrm{T}))}/{\sigma X}$$
when I know that the term inside the exponent is essentially a scalar. Should I differentiate according to
$$\mathrm{e}^{-A} \equiv \sum_{n=0}^\infty \frac1{n!}(-1)^n A^n = I - A + \frac12 A^2 - \cdots$$
or should I use regular exponent differentiation rules and to apply matrix differentiation only on the term $a\cdot X^{-1} \cdot a^\mathrm{T}$ using $$\frac{\partial a^\mathrm{T} X^{-1} b}{\partial X} = -X^{-\mathrm{T}}ab^{\mathrm{T}}X^{-\mathrm{T}}\ ?$$
You want to calculate the derivative of $f = {\rm{exp}}(y)$, where $y = a^TX^{-1}b$.
You already know the derivative $y$ -- so apply the chain rule: $$ \eqalign { \frac {\partial f} {\partial X} &= \frac {\partial f} {\partial y} \frac {\partial y} {\partial X} \cr &= f \frac {\partial y} {\partial X} \cr &= -X^{-T} a\,b^T X^{-T}\,\,{\rm{exp}}(a^TX^{-1}b) \cr }$$