Partial derivative for Gaussian Smoothing

Question

$G = (2\pi)^{-d/2} \int\limits_{\mathbb{R}^d} F(\theta + \sigma \epsilon) e^{-\frac{1}{2}||\epsilon||^2}d\epsilon$

$\frac{\partial G}{\partial \sigma} = (2\pi)^{-d/2} (\frac{1}{\sigma}) \int\limits_{\mathbb{R}^d} F(\theta + \sigma \epsilon) e^{-\frac{1}{2}||\epsilon||^2} \epsilon d\epsilon$

Can someone show me the steps?

Do I need the chain rule and integration by parts?

Source: https://arxiv.org/abs/1804.02395 Page 2

Answer 1

Using $\delta = \sigma \epsilon$ we have that

$$G = (2\pi\sigma^2)^{-\frac{d}{2}} \int_{\mathbb{R}^d} F(\theta+\delta) e^{-\frac{||\delta||^2}{2\sigma^2}} d\delta$$

$$\frac{dG}{d\sigma} = -\frac{d}{\sigma}(2\pi\sigma^2)^{-\frac{d}{2}} \int_{\mathbb{R}^d} F(\theta+\delta) e^{-\frac{||\delta||^2}{2\sigma^2}} d\delta + (2\pi\sigma^2)^{-\frac{d}{2}}\int_{\mathbb{R}^d}F(\theta+\delta)e^{-\frac{||\delta||^2}{2\sigma^2}}\frac{||\delta||^2}{\sigma^3}d\delta$$

Then undo the change of variables

$$\frac{dG}{d\sigma} = -\frac{d}{\sigma}(2\pi)^{-\frac{d}{2}}\int_{\mathbb{R}^d} F(\theta+\sigma\epsilon)e^{-\frac{||\epsilon||^2}{2}}d\epsilon + \frac{1}{\sigma}(2\pi)^{-\frac{d}{2}}\int_{\mathbb{R}^d} F(\theta+\sigma\epsilon)e^{-\frac{||\epsilon||^2}{2}}||\epsilon||^2d\epsilon$$

$$\implies \frac{dG}{d\sigma} = (2\pi)^{-\frac{d}{2}}\left(\frac{1}{\sigma}\right)\int_{\mathbb{R}^d} F(\theta+\sigma\epsilon)(||\epsilon||^2-d)e^{-\frac{||\epsilon||^2}{2}}d\epsilon$$

But this is not what the paper is saying. The paper correctly states that the gradient of $G$ with respect to $\theta$, not $\sigma$, is what you have. The proof is similar, except now take $\delta = \theta + \sigma\epsilon$:

$$G(\theta) = (2\pi\sigma^2)^{-\frac{d}{2}} \int_{\mathbb{R}^d} F(\delta) e^{-\frac{||\delta-\theta||^2}{2\sigma^2}}d\delta$$

$$\nabla G(\theta) = (2\pi\sigma^2)^{-\frac{d}{2}} \int_{\mathbb{R}^d} F(\delta) e^{-\frac{||\delta-\theta||^2}{2\sigma^2}}\cdot\left(\frac{\delta-\theta}{\sigma^2}\right)d\delta$$

Then undo the change of variables to get

$$\nabla G(\theta) = (2\pi)^{-\frac{d}{2}}\left(\frac{1}{\sigma}\right) \int_{\mathbb{R}^d} F(\theta+\sigma\epsilon) e^{-\frac{||\epsilon||^2}{2}}\epsilon \:d\epsilon$$

as desired.