Consider $q(\boldsymbol{x}) \sim \mathcal{N}(\boldsymbol{\mu}, \boldsymbol{\Sigma})$, I need to compute the derivatives of $\mathbb{E}_q[V(x)]$ on $\mu$ and $\Sigma$. The results are expected as $\triangledown_{\boldsymbol{\mu}} \mathbb{E}_q[V(x)] = \mathbb{E}_q[\triangledown_{\boldsymbol{x}} V(x)]$ and $\triangledown_{\boldsymbol{\Sigma}} \mathbb{E}_q[V(x)] = \frac{1}{2} \mathbb{E}_q[\triangledown_{\boldsymbol{x}} \triangledown_{\boldsymbol{x}} V(x)]$, but I cannot figure out how to get them.
The Appendix A in the paper (http://www0.cs.ucl.ac.uk/staff/c.archambeau/publ/neco_mo09_web.pdf) provides a solution using characteristic functions, but the intermediate steps are missing. Could anyone give me some help? Thanks in advance!:)
(1) Proof of $\triangledown_\mu \mathbb{E}_q[V(x)] = \mathbb{E}_q[\triangledown_xV(x)]$:
By definition of characteristic functions, we can write the $\mathbb{E}_q[V(x)]$ as $$ \begin{align*} \mathbb{E}_q[V(x)] =& \frac{1}{(2 \pi)^n} \int e^{-\frac{1}{2}k^T\Sigma k + ik(\mu-x)}V(x)dxdk. \end{align*} $$ We compute its derivative to $\mu$ by switching the derivative and integration, and get $$ \triangledown_\mu \mathbb{E}_q[V(x)] = \frac{1}{(2 \pi)^n} \int e^{-\frac{1}{2}k^T\Sigma k + ik(\mu-x)}V(x) (ik)dxdk. $$ We compute its derivative to $x$ in the same way, and get $$ \begin{align*} \triangledown_x \mathbb{E}_q[V(x)] =& \frac{1}{(2 \pi)^n} \int e^{-\frac{1}{2}k^T\Sigma k + ik(\mu-x)}V(x) (-ik) dxdk\\ &+\frac{1}{(2 \pi)^n} \int e^{-\frac{1}{2}k^T\Sigma k + ik(\mu-x)} \triangledown_x V(x)]dxdk \\ =&-\triangledown_\mu \mathbb{E}_q[V(x)] + \mathbb{E}_q[\triangledown_xV(x)]. \end{align*} $$ If we have $\triangledown_x \mathbb{E}_q[V(x)] = 0$, we will get $\triangledown_\mu \mathbb{E}_q[V(x)] = \mathbb{E}_q[\triangledown_xV(x)]$.
(2) Proof of $ \triangledown_\Sigma \mathbb{E}_q[V(x)] = \frac{1}{2}\mathbb{E}_q[\triangledown_x \triangledown_x V(x)] = \frac{1}{2} \mathbb{E}_q[\triangledown_\mu \triangledown_\mu V(x)]$
We compute the derivative to $\Sigma$ as before, and get $$\triangledown_\Sigma \mathbb{E}_q[V(x)] = \frac{1}{(2 \pi)^n} \int e^{-\frac{1}{2}k^T\Sigma k + ik(\mu-x)}V(x) (-\frac{1}{2}k^Tk)dxdk.$$ By proving (1), we already have $\triangledown_\mu \mathbb{E}_q[V(x)] = \mathbb{E}_q[\triangledown_xV(x)]$, considering $\triangledown_x \mathbb{E}_q[V(x)] = 0$. We substitute $\triangledown_x V(x)$ in place of $V(x)$, and get $\triangledown_\mu \mathbb{E}_q[\triangledown_x V(x)] = \mathbb{E}_q[\triangledown_x \triangledown_x V(x)]$, since we did not restrict the form on the $V(x)$. In this way, we "move" the $\triangledown_\mu$ into the expectation.
We can also "move" the $\triangledown_x$ out of the expectation, following the result of (1), and get $$ \begin{align*} \triangledown_\mu \mathbb{E}_q[\triangledown_x V(x)] =& \triangledown_\mu \triangledown_\mu \mathbb{E}_q[V(x)] \\ =& \frac{1}{(2 \pi)^n} \int e^{-\frac{1}{2}k^T\Sigma k + ik(\mu-x)}V(x) (-k^Tk)dxdk\\ =&2 \triangledown_\Sigma \mathbb{E}_q[V(x)] . \end{align*} $$ Therefore, $ \triangledown_\Sigma \mathbb{E}_q[V(x)] = \frac{1}{2}\mathbb{E}_q[\triangledown_x \triangledown_x V(x)] = \frac{1}{2} \mathbb{E}_q[\triangledown_\mu \triangledown_\mu V(x)]$.