$L^2$ distance between Gaussian probability density functions

Question

In order to understand different ways of comparing probability functions, I'm trying to bound from above the $L^2$ distance between the pdf of two multivariate Gaussians. More specifically, let $f,f' : \mathbb{R}^n \to \mathbb{R}$ be the pdf of two multivariate Gaussians with means $\mu,\mu' \in \mathbb{R}^n$, respectively, and with the same covariance matrix $\Sigma$. My intuition tells me that $$ \|f - f'\|_2 \leq \alpha \|\mu - \mu'\|_2, $$ for some constant $\alpha$ that only depends on $\Sigma$. However, I'm not even sure of how to start. I have tried considering the differential of the quantity $\|f - f'\|_2$ with respect to $\mu'$, but I got stuck fairly quickly as I am not too familiar with taking derivatives under a norm.

Answer 1

Let $\displaystyle\langle f,g \rangle_{2} := \int_{\mathbb{R}^n} f(x) g(x) \ \mathrm{d}x$ denote the inner product over the space of $L^2$ functions. For notational convenience, we define an inner product over $\mathbb{R}^n$ as $\langle x,y \rangle_{\Sigma} = x^{\top} \Sigma^{-1} y$ for any $x,y \in \mathbb{R}^n$. Let $\|x\|_{\Sigma} = \sqrt{\langle x,x \rangle_{\Sigma}}$ denote the associated norm. With the notation in order, note that $$ \|f - f'\|_2^2 = \|f\|_2^2 + \|f'\|_2^2 - 2 \langle f,f' \rangle_{2}.$$

We have, \begin{align*} \|f\|^2_2 & = \int_{\mathbb{R}^n} (2\pi)^{-n} \det(\Sigma)^{-1} \exp \left(- \|x- \mu\|_{\Sigma}^2\right) \ \mathrm{d}x \\ & = (2\pi)^{-n} \det(\Sigma)^{-1} \cdot (4\pi)^{n/2} \det(\Sigma)^{1/2} \\ & = \pi^{-n/2} \det(\Sigma)^{-1/2}. \end{align*} Since the above expression does not depend on $\mu$, we also have $\|f'\|_2^2 = \pi^{-n/2} \det(\Sigma)^{-1/2}$. For the third term, we have, \begin{align*} \langle f, f' \rangle_{2} & = \int_{\mathbb{R}^n} (2\pi)^{-n} \det(\Sigma)^{-1} \exp \left(- \frac{1}{2}\|x- \mu\|_{\Sigma}^2 - \frac{1}{2}\|x- \mu'\|_{\Sigma}^2\right) \ \mathrm{d}x \\ & = \int_{\mathbb{R}^n} (2\pi)^{-n} \det(\Sigma)^{-1} \exp \left(- \|x\|_{\Sigma}^2 + \langle x, \mu + \mu' \rangle_{\Sigma} - \frac{1}{2}\| \mu\|_{\Sigma}^2 - \frac{1}{2}\| \mu'\|_{\Sigma}^2\right) \ \mathrm{d}x \\ & = \int_{\mathbb{R}^n} (2\pi)^{-n} \det(\Sigma)^{-1} \exp \left(- \left\|x - \frac{\mu + \mu'}{2} \right\|_{\Sigma}^2 - \left\|\frac{\mu - \mu'}{2} \right\|_{\Sigma}^2\right) \ \mathrm{d}x \\ & = \exp \left(- \left\|\frac{\mu - \mu'}{2} \right\|_{\Sigma}^2\right)\int_{\mathbb{R}^n} (2\pi)^{-n} \det(\Sigma)^{-1} \exp \left(- \left\|x - \frac{\mu + \mu'}{2} \right\|_{\Sigma}^2 \right) \ \mathrm{d}x \\ & = \exp \left(- \left\|\frac{\mu - \mu'}{2} \right\|_{\Sigma}^2\right) \cdot \pi^{-n/2} \det(\Sigma)^{-1/2}. \end{align*} In the last step above, we used the same steps as used for $\|f\|_2^2$. On combining all of them, we obtain, \begin{align*} \|f - f'\|_2^2 & = 2\pi^{-n/2} \det(\Sigma)^{-1/2} \left( 1 - \exp \left(- \left\|\frac{\mu - \mu'}{2} \right\|_{\Sigma}^2\right)\right)\\ & \leq 2\pi^{-n/2} \det(\Sigma)^{-1/2} \left\|\frac{\mu - \mu'}{2} \right\|_{\Sigma}^2\\ & \leq \|\Sigma^{-1}\|_2 \cdot 2\pi^{-n/2} \det(\Sigma)^{-1/2} \left\|\frac{\mu - \mu'}{2} \right\|^2, \end{align*} where $\|A\|_2$ denotes the spectral norm of $A$. On taking the square root, you arrive at the result.