I'm a bit stuck evaluating the relative entropy $\int_{}^{} f(\textbf{x}) \log \left(\tfrac{f(\textbf{x})}{g(\textbf{x})} \right) \mathrm{d}\textbf{x}$ (where f and g are two densities) in the case given below and I would appreciate someone's help:
$X_{1}$ has pdf N($\mu_{1}$,$\sigma^{2}$) and $X_{2}$ has pdf N($\mu_{2}$,$\sigma^{2}$)
If $f_1$ is the $N(\mu_1,\Sigma_1)$ and $f_2$ is the $N(\mu_2,\Sigma_2)$ pdf, note that $f_i (x) = \frac{1}{(2 \pi)^{p/2} |\Sigma_i|^{1/2}} e^{-\frac{1}{2}(x-\mu_i)^T \Sigma_i^{-1} (x-\mu_i)}$.
Now, $D(f_1 || f_2) = \int_{\mathbb{R}^p} f_1 (x) \log \frac{f_2}{f_1}(x) = \frac{1}{2} \int_{\mathbb{R}^p} f_1(x) \left(\log \frac{|\Sigma_1|}{|\Sigma_2|} -(x-\mu_1)^T \Sigma_1^{-1} (x-\mu_1) +(x-\mu_2)^T \Sigma_2^{-1} (x-\mu_2) \right)$
Multiply this out to get terms of the form constants (i.e. don't contain $x$), $x^T v$, $v^T x$ and $x^T \Sigma_i x$ [all times $f_1$] in the integrand.
You'll get a bunch of terms which are of the form $\int x^T v f_1(x)$ for a vector $v$ (or $\int v^T x f_1(x)$ -- $v$ will depend typically on $\mu_i$ and $\Sigma_i^{-1}$. These will simply be $\mu_1^T v$ or $v^T \mu_1$ by linearity of expectation. Note that $\mu_i^T \Sigma_i^{-1} \mu_i$ is a constant as is $log \frac{|\Sigma_1|}{|\Sigma_2|} $ so you can pull those out and use the fact that $\int f_1 = 1$ to handle those terms.
The remaining terms $\int x^T \Sigma_i^{-1} x f_1(x) = E_1 [ x^T \Sigma_i^{-1} x] $ are the problem. Here, the trace permutation trick is useful. Note that $x^T \Sigma_i^{-1} x = tr(x^T \Sigma_i^{-1} x) = tr(x x^T \Sigma_i^{-1})$. Thus, $E_1 [ x^T \Sigma_i^{-1} x] = E_1[tr(x^T \Sigma_i^{-1} x)] = E_1[tr(x x^T \Sigma_i^{-1})] = tr(E_1[x x^T] \Sigma_i^{-1}) = tr( (\Sigma_1 + \mu_1 \mu_1^T) \Sigma_i^{-1})$ by linearity of expectation and definition of covariance matrix.