Is it obvious that this integral converges given the following assumptions?

Question

The integral is $\int\limits_{p(x) > 0}p^{-\lambda + 1}(x) \, \left| \ln p(x) \right|^k \, dx$.

Assumptions:

• $\lambda > 0, k > 0$
• $\int\limits_{p(x) > 0}p^{-\lambda + 1}(x) \, dx < \infty$
• $p$ is a probability density function satisfying $\int p(x) \, dx = 1$.

This comes from the first few paragraphs in this paper on a topic in information theory -- http://arxiv.org/pdf/1204.3661.pdf (see eq. 2.4), which states that "it is not hard to see that ..." The paper explains the exact context in a few lines around eq. 2.1-2.5. The claim doesn't seem obvious to me. In fact, I am not 100% confident it is even always true. Perhaps I'm missing an obvious connection that makes this clear ... Please view the specified portion of the reference before attempting to answer.

Answer 1

Below is the detailed proof of the integral $\int\limits_{p(x)>0} p^{-\lambda+1}(x) \| \ln p(x) \|^k dx$ is finite for any positive integer k under the assumption that

1. $0 < \lambda < \lambda^*$ where $\lambda^*$ is defined as $\sup\{ \lambda \geq 0 : \int p^{-\lambda+1}(x) dx < +\infty\}$
2. and $p(x)$ is a probability density function, i.e. $\int p(x) dx = 1$.

According to the definition of $\lambda^*$, for any $\lambda$ strictly less than $\lambda^*$, $\int p^{-\lambda+1}(x) dx < +\infty$. Also, the claim $\int\limits_{p(x)>0} p^{-\lambda+1}(x) \| \ln p(x) \|^k dx$ is finite is only for any $\lambda$ strictly less than $\lambda^*$.

First, we can show that $B(\lambda) = \int\limits_{p(x) \geq 1} p^{-\lambda+1}(x) dx$ and $ C(\lambda) = \int\limits_{0 < p(x) < 1} p^{-\lambda+1}(x) dx$ are both finite for $0 < \lambda < \lambda^*$. Let $A (\lambda) = \int\limits_{p(x)>0} p^{-\lambda+1}(x) dx$ be a fintie number depending on $\lambda$.

$$ B(\lambda) + C(\lambda) = A(\lambda)$$ $$ 0 \leq B(\lambda) = \int\limits_{p(x) \geq 1} p^{-\lambda+1}(x) dx = \int\limits_{p(x) \geq 1} p^{-\lambda}(x) p(x) dx \leq \int\limits_{p(x) \geq 1} p(x) dx \leq \int\limits_{p(x) > 0} p(x) dx = 1$$

Therefore,

$$ A(\lambda)-1 \leq C(\lambda) \leq A(\lambda)$$

Now given a $\lambda$ and a $k$, notice that $\ln y \leq \frac{1}{\alpha} y^{\alpha}$ when $y \geq 1$ and $\ln 1/y \leq \frac{1}{\alpha} y^{-\alpha}$ when $0 < y < 1$ for any positive $\alpha$. First, let $\alpha = \frac{\lambda}{2k}$. Consequently,

\begin{align} \int\limits_{p(x) \geq 1} p^{-\lambda+1}(x) \| \ln p(x) \|^k dx &\leq \int\limits_{p(x) \geq 1} p^{-\lambda+1}(x) \left( \frac{2k}{\lambda} \right)^k p^{\frac{\lambda}{2}}(x) dx \\ &= \left( \frac{2k}{\lambda} \right)^k \int\limits_{p(x) \geq 1} p^{-\frac{\lambda}{2}+1}(x) dx \\ &= \left( \frac{2k}{\lambda} \right)^k B(\lambda/2) \end{align}

Then, let $\alpha = \frac{\lambda^* - \lambda}{2k}$,

\begin{align} \int\limits_{0 < p(x) < 1} p^{-\lambda+1}(x) \| \ln p(x) \|^k dx &= \int\limits_{0 < p(x) < 1} p^{-\lambda+1}(x) \left( \frac{2k}{\lambda^* - \lambda} \right)^k \ln^{k} \frac{1}{p(x)} dx \\ &\leq \left( \frac{2k}{\lambda^* - \lambda} \right)^k \int\limits_{0 < p(x) < 1} p^{-\lambda+1}(x) p^{-\frac{\lambda^* - \lambda}{2}}(x) dx \\ &= \left( \frac{2k}{\lambda^* - \lambda} \right)^k \int\limits_{0 < p(x) < 1} p^{-\frac{\lambda^*+\lambda}{2}+1}(x) dx \\ &= \left( \frac{2k}{\lambda^* - \lambda} \right)^k C \left(\frac{\lambda^*+\lambda}{2}\right) \end{align}

Consequently,

$$\int\limits_{p(x)>0} p^{-\lambda+1}(x) \| \ln p(x) \|^k dx \leq \left( \frac{2k}{\lambda} \right)^k B(\lambda/2) + \left( \frac{2k}{\lambda^* - \lambda} \right)^k C\left(\frac{\lambda^*+\lambda}{2}\right)$$

Since $0 < \lambda < \lambda^*$, so are $\frac{\lambda}{2}$ and $\frac{\lambda^*+\lambda}{2}$, and therefore both $B(\lambda/2)$ and $C(\frac{\lambda^*+\lambda}{2})$ are finite. And the proof is finished.

Notice that the claim never mentions what whould happen if $\lambda = \lambda^*$. In fact, the counter example just shows that $\int\limits_{p(x)>0} p^{-\lambda+1}(x) \| \ln p(x) \|^k dx$ can be unbounded when $\lambda = \lambda^*$. Indeed, it can be shown that $\lambda^* = 2$ for $p(x)$ given by the counter example. To see that, just try to bound $\int\limits p^{-\lambda+1}(x) dx$ for any $\lambda > 2$ and you will see that no matter how close $\lambda$ is with $2$, $\int\limits p^{-\lambda+1}(x) dx$ is always unbounded.

Answer 2

Counterexample: Define $p(x)= x(\ln x)^2$ in an interval $[0,b]$ for some small $b>0,$ constant on some $[b,c]$ so that $\int_0^c p(x)\,dx =1,$ and then is $0$ for other $x.$

Let $\lambda = 2, k =1.$ Then

$$\int_0^c p^{-1}(x)\, dx = \int_0^b \frac{1}{x(\ln x)^2}\, dx + \int_b^c \frac{1}{p(b)}\, dx < \infty.$$

So $p$ satisfies the hypotheses. Claim:

$$ \int_0^{b} p(x)^{-1}\cdot |\ln p(x)|\,dx = \infty.$$

Indeed, the integrand is

$$\frac{|\ln x + 2\ln (|\ln x|)| }{x(\ln x)^2} \sim \frac{1}{x|\ln x|}$$ near $0.$ That's a nonintegrable singularity, proving the claim.

---

Prvious incorrect answer: It doesn't look good: Take $p(x) = x, x \in (0,1), 0 $ elsewhere, $\lambda = 2, k =1.$

Answer 3

1. I would give an explanation about the following claim in the reference: Under the assumption $\lambda^*(X)>0$, for any $\lambda\in (0,\lambda^*(X))$ and any positive (integer) $k$, the following integrals are finite, \begin{gather} \int p^{-\lambda+1}(x)\,dx<\infty, \tag{1}\\ \int p^{-\lambda+1}(x)|\ln p(x)|^k\,dx<\infty \tag{2} \end{gather} According the definition of $\lambda^*(X)\triangleq \sup\{\lambda\ge 0:\int p^{-\lambda+1}(x)\,dx<\infty \}$, it suffices to prove that (2) is true.

Now we use following inequality (which can be derived from $1+y\le e^y$) , $$|\log p|^k\le\begin{cases}C_{k,\alpha}p^\alpha,\quad& p\ge 1,\\ C_{k,\alpha}\frac1{p^\alpha}, & 0<p<1. \end{cases} \qquad k,\alpha >0 \tag{3}$$ where $C_{k,\alpha}$ is a constant depending on $k$ and $\alpha$(for example, $C_{k,\alpha}=(\frac{k}{\alpha e})^k$). For $\lambda\in (0,\lambda^*(X))$, there exists a $\lambda_2\in(\lambda, \lambda^*(X))$ such that $$\int [p(x)]^{-\lambda_2+1}\,dx<\infty. \tag{4}$$ Meanwhile, using (3), we have \begin{gather} \int p^{-\lambda+1}(x)|\ln p(x)|^k1_{\{y:p(y)\ge1\}}(x)\,dx\le C_{k,\lambda}\int p(x)\,dx<\infty,\\ \int p^{-\lambda+1}(x)|\ln p(x)|^k1_{\{y:p(y)<1\}}(x)\,dx\le C_{k,\lambda_2-\lambda} \int [p(x)]^{-\lambda_2+1}\,dx<\infty,\\ \end{gather} Now (2) is obtained from above inequalities.

2. To get $C_{k,\alpha}$: \begin{align} &1+y\le e^y,\quad \forall y\in\mathbb{R} \qquad \Rightarrow \qquad x\le e^{x-1}=\frac1e e^x, \quad \forall x\in\mathbb{R}\\ \Rightarrow\quad & \frac{\alpha x}{k}\le \frac1e e^{(\alpha x)/k}\qquad \Rightarrow\quad \Bigl(\frac{\alpha x}{k}\Bigr)^k\le \frac1{e^k}e^{\alpha x},\quad \alpha,k,x>0\\ \Rightarrow\quad & x^k\le \Bigl(\frac{k}{\alpha e}\Bigr)^k e^{\alpha x},\quad \alpha,k,x>0 \qquad \Rightarrow\quad (3) \end{align}

3. The counterexample given by zhw in the previous answer is correctly to provide an example to interpret the following claim couldn't be true: $$ \left .\begin{matrix}\int_{p(x)>0}p(x)^{-\lambda+1}\,dx<\infty,\\ \lambda>0, k>0,\\ \int p(x)\,dx=1 \end{matrix}\right\} \Rightarrow\int_{p(x)>0}p^{-\lambda+1}(x)|\ln p(x)|^k\,dx<\infty. \tag{5} $$ The (5) is different from the following statement: $$ \left .\begin{matrix}\int p(x)\,dx=1\\ \lambda_2>\lambda>0, k>0\\ \int_{p(x)>0}p(x)^{-\lambda_2+1},dx<\infty, \end{matrix}\right\} \Rightarrow\int_{p(x)>0}p^{-\lambda+1}(x)|\ln p(x)|^k\,dx<\infty \tag{6} $$ (6) is a claim derived from the reference and proved in the above.

Answer 4

You have not written down the condition correctly. Rather than $$\int p^{\lambda-1}(x)dx<\infty$$ it should read $$\int p^{\lambda^*-1}(x)dx<\infty\quad\text{for some }\lambda^*>\lambda.$$ These conditions are not equivalent, and in fact the statement in the paper is correct (as shown in other answers) whereas under your assumptions the statement is not correct (see the counterexample provided by zhw). Be careful with your assumptions.