I keep messing up my indices and not getting the right answer. I want to compute
$$ \dfrac{d}{dJ_i} \dfrac{d}{dJ_j}\exp\left( \dfrac{1}{2}\mathbb{J}\cdot\mathbb{A}^{-1} \cdot\mathbb{J} \right)=~? $$
For this notation $\mathbb{J}$ can be written $J_\lambda$ and $\mathbb{A}^{-1}$ (which is symmetric) can be written $A^{-1}_{\lambda\rho}$. We have necessarily
$$ \dfrac{d}{dJ_i}\left(\mathbb{J}\cdot x\right)=\dfrac{d}{dJ_i}\sum J_\lambda x_\lambda=x_i $$
Therefore, I want to rewite the main equation which is giving me problems as
$$ \dfrac{d}{dJ_i} \dfrac{d}{dJ_j}\exp\left( \dfrac{1}{2}\sum\sum J_\lambda A^{-1}_{\lambda\rho}J_\rho \right)=~? $$
However, when I start to doing the chain rule I end up with too many terms that don't seem to cancel. I guess, the simplest way to ask my question is if I am doing this derivative correctly.
$$ \dfrac{d}{dJ_j}\left( \dfrac{1}{2}\sum\sum J_\lambda A^{-1}_{\lambda\rho}J_\rho \right)=\dfrac{1}{2} \left( \sum A^{-1}_{j\rho}J_\rho + \sum J_\lambda A^{-1}_{\lambda j} \right)=\sum A^{-1}_{j\rho}J_\rho $$
Is that right ($A^{-1}$ is symmetric)? My thinking, which seems to be wrong because if I stopped thinking it then I would get the right answer right away, is that $d/dJ_\alpha$ needs to take the derivative like
$$ \dfrac{d}{dJ_i} \dfrac{d}{dJ_j}\exp\left( \dfrac{1}{2}\sum\sum J_\lambda A^{-1}_{\lambda\rho}J_\rho \right)=$$
\begin{align} &=\dfrac{d}{dJ_i} \exp\left( \dfrac{1}{2}\sum\sum J_\lambda A^{-1}_{\lambda\rho}J_\rho \right)\dfrac{d}{dJ_j}\left( \dfrac{1}{2}\sum\sum J_\lambda A^{-1}_{\lambda\rho}J_\rho \right)\\ &=\dfrac{d}{dJ_i} \exp\left( \dfrac{1}{2}\sum\sum J_\lambda A^{-1}_{\lambda\rho}J_\rho \right) \sum A^{-1}_{j\rho}J_\rho\\ &=\exp\left( \dfrac{1}{2}\sum\sum J_\lambda A^{-1}_{\lambda\rho}J_\rho \right)\dfrac{d}{dJ_i}\left( \dfrac{1}{2}\sum\sum J_\lambda A^{-1}_{\lambda\rho}J_\rho \right)\left( \sum A^{-1}_{j\rho}J_\rho \right)+\\ &\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad+ \exp\left( \dfrac{1}{2}\sum\sum J_\lambda A^{-1}_{\lambda\rho}J_\rho \right)\dfrac{d}{dJ_i}\left( \sum A^{-1}_{j\rho}J_\rho \right)\\ &= \exp\left( \dfrac{1}{2}\sum\sum J_\lambda A^{-1}_{\lambda\rho}J_\rho \right)\left( \sum A^{-1}_{i\rho}J_\rho \right)\left( \sum A^{-1}_{j\rho}J_\rho \right)+ \exp\left( \dfrac{1}{2}\sum\sum J_\lambda A^{-1}_{\lambda\rho}J_\rho \right)\left( A^{-1}_{ji} \right) \end{align}
...etc. I think I have already gotten an error by the time I have this many terms. The answer say the derivative should bring down $A^{-1}_{ij}$ which is my second term, but I have this first term too which, I guess, should disappear when I don't do it wrong. Would some kind person please point out my error or take this derivative for me step by step? Thanks :)
Thanks in advance.
Let's use the convention where uppercase Latin represent matrices, lowercase Latin vectors, and Greeks are scalars.
Then let's use symbols which are easier to type $$\eqalign{ B &= B^T=A^{-1} \cr x &= {\mathbb J} \cr \mu &= \tfrac{1}{2}x^TBx &\implies d\mu = (Bx)^Tdx \cr \alpha &= \exp(\mu) &\implies d\alpha = \alpha\,d\mu \cr }$$ Let's continue the expansion of $d\alpha$ and find its gradient $$\eqalign{ d\alpha &= \alpha(Bx)^Tdx \cr g &= \frac{\partial\alpha}{\partial x} = \alpha Bx \cr }$$ Now find the differential and gradient of $g$ $$\eqalign{ dg &= Bx\,d\alpha + \alpha B\,dx \cr &= Bx\,\alpha(Bx)^Tdx + \alpha B\,dx \cr &= \alpha Bxx^TB\,dx + \alpha B\,dx \cr H &= \frac{\partial g}{\partial x} = \alpha(B+Bxx^TB) \cr }$$ Finally, setting $\,x=0\,$ yields $\,\alpha = e^0=1\,$ and $\,H = B$.