Integral with logarithm is positive

Question

Given the following integral:

$$I(f) = \int_\mathbb{R} f(x) \log \left(f(x) \sqrt{2\pi} e^{\frac{x^2}{2}}\right) dx,$$ where we assume $\int_{\mathbb{R}} f(x)\, dx =1$ and $f\geq 0$ a.e. Assume for convenience that $f$ is decaying fast enough to make any integration working.

I want to show now that $I(f)\geq 0$ using the hint $x\log x \geq x -1$ for $x\geq 0$.

Answer 1

So, try to only look at the first line in the calculation below and finish it yourself. If you do not succeed, look at the other lines... $$ \begin{aligned} I(f)&=\int_{\mathbb{R}}\frac{1}{\sqrt{2\pi}}e^{-x^2/2}\bigl(f(x)\sqrt{2\pi}e^{x^2/2}\bigr)\log \bigl(f(x)\sqrt{2\pi}e^{x^2/2}\bigr)\,dx\\ &\geq\int_{\mathbb{R}}\frac{1}{\sqrt{2\pi}}e^{-x^2/2}\Bigl[f(x)\sqrt{2\pi}e^{x^2/2}-1\Bigr]\,dx\\ &=\int_{\mathbb{R}}f(x)\,dx-\int_{\mathbb{R}}\frac{1}{\sqrt{2\pi}}e^{-x^2/2}\,dx\\ &=0. \end{aligned} $$

Answer 2

Note that $f$ is a probability distribution function and $I(f)=\int_{\mathbb{R}}\frac{1}{2\pi}e^{-x^2/2}g\ln g dx$ where $g(x)=f(x) \sqrt{2\pi} e^{\frac{x^2}{2}}$. Since $g\ge 0$ a.e., according to the given hint, $$I(f)\ge \int_{\mathbb{R}}f dx-\int_{\mathbb{R}}\frac{1}{2\pi}e^{-x^2/2}dx=0$$ since the second part is the area under the probability distribution of a normal random variable.

Comment: The given expression is actually the relative entropy or the Kullback-Liebler distance of the normal density function from the density function $f(\cdot)$. This function can be shown to be positive for arbitrary density functions which are defined over the same support.