I would like to know when it is allowed to interchange derivation and expectation. Suppose $X$ is some r.v whose dynamic is controlled by some parameter $\sigma$ and suppose $h$ is some smooth function of two variables. Is the following true: $$\frac{\partial \mathbb{E}\left[h(X,\sigma)\right]}{\partial \sigma} =\mathbb{E}\left[\frac{\partial h(X,\sigma)}{\partial \sigma} \right]$$
I think it's wrong (the expectation operator depends on $\sigma$ through the dynamics of $X$). If $d\mathbb{P}_X(x)=p_{X,\sigma}(x)dx$, then we would have $$\frac{\partial \mathbb{E}\left[h(X,\sigma)\right]}{\partial \sigma} =\int \frac{\partial}{\partial \sigma}\left[h(X,\sigma)p_{X,\sigma}(x)\right]dx$$ which is in general different from $$\mathbb{E}\left[\frac{\partial h(X,\sigma)}{\partial \sigma} \right] =\int \frac{\partial h(X,\sigma)}{\partial \sigma}p_{X,\sigma}(x)dx$$
Could somebondy confirm my reasoning ?
PS. I posted this on Quant SE and somebody advised to post it on Math SE.
This is not always true. For a counterexample, take $X \sim N(\mu, \sigma^2)$, a normally distributed random variable with mean $\mu$ and variance $\sigma^2$, and define $h$ as
$$h(X, \sigma) = \left(\frac{X- \mu}{\sigma}\right)^2$$
We have
$$\frac{\partial \mathbb{E}\left[h(X,\sigma)\right]}{\partial \sigma}= \frac{\partial}{\partial \sigma}(1) = 0$$
and
$$\mathbb{E}\left[\frac{\partial h(X,\sigma)}{\partial \sigma} \right]= \mathbb{E}\left[\frac{-2(X- \mu)^2}{\sigma^3} \right] = -\frac{2}{\sigma}\mathbb{E}\left[\left(\frac{X- \mu}{\sigma}\right)^2 \right]= -\frac{2}{\sigma}$$