What is the KL divergence $KL(P \;\Vert\; Q)$ between an Exponential distribution $P = \text{Exp}(\lambda)$ and a Normal distribution $Q = \mathcal N(\mu, \sigma^2)$?
I have not found any source for this problem and because the integration by hand is difficult, I would like to have some confirmation.
The integral is given by
$$KL(P \;\Vert\; Q) = \int_0^\infty \lambda e^{-\lambda x} \log\Biggl( \frac{\lambda e^{-\lambda x}\sqrt{2\pi\sigma^2}}{e^{-\frac{(x-\mu)^2}{2\sigma^2}}} \Biggr) dx$$
As a little bit of short hand let $$ \langle h(X) \rangle = \mathbb{E}[h(X)] = \int h(X)p(x)dx, $$ so $$ \begin{align} D_{KL}(P \| Q) &= \langle \log p(X) \rangle -\langle \log q(X) \rangle, \end{align} $$ Then considering seperately you need to do the integrals $$ \langle \log p(X) \rangle = \langle \log \lambda -\lambda X\rangle =\log \lambda - \lambda \langle X\rangle, $$ and $$ \begin{align} \langle \log Q(X) \rangle &= \left\langle \log \left(\frac{1}{\sqrt{2\pi \sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}\right)\right\rangle \\ &= -\frac{1}{2\sigma^2}\langle(X-\mu)^2\rangle -\frac{1}{2}\log (2\pi\sigma^2), \end{align} $$ which assuming I haven't blundered along the way should be easy enough for you to continue onwards. One check/interesting result is that minimising $D_{KL}(P\| Q)$ with respect to the parameter $(\mu, \sigma^2)$ should be equivalent to moment matching.