Geometric Series to approximate power series expansion

Question

I have been stuck working through the following derivation for a while now (for context the problem is a Central Limit Theorem proof in Stochastic Processes: Theory For Applications, specifically the proof of Theorem 1.7.3).

Problem

The specific issue I am having is in going from (1.87) to (1.88) in the following (quoting directly from the text):

Let $\epsilon = \hat{p} - p$ and $q = 1 - p$. Then: $$ S = (p + \epsilon)ln(1 + \frac{\epsilon}{p}) + (q - \epsilon)ln(1 - \frac{\epsilon}{q}) $$ $$ S = \frac{\epsilon^2}{2p} - \frac{\epsilon^3}{6p^2} + \dots + \frac{\epsilon^2}{2q} + \frac{\epsilon^3}{6q^2} + \dots \tag{1.85}\label{1.85} $$ $$ S = \frac{\epsilon^2}{2pq} - \frac{\epsilon^3}{6p^2} + \dots + \frac{\epsilon^3}{6q^2} + \dots \tag{1.86}\label{1.86} $$ In (1.86) we used the fact that $\frac{1}{p}+\frac{1}{q} = \frac{1}{pq}$. Substituting this into an exponential term: $$exp(-nS) = exp\big(-\frac{n\epsilon^2}{2pq} \big) exp\big( \frac{\epsilon^3}{6p^2} + \dots - \frac{\epsilon^3}{6q^2} + \dots\big)\tag{1.87}\label{1.87}$$ $$exp(-nS) = exp\big(-\frac{n\epsilon^2}{2pq} \big) (1 \pm C_2 n^{-3\alpha + 2}) \tag{1.88}\label{1.88}$$ Where in (1.88) we used, first, the fact that the neglected terms in (1.87) are decreasing at least geometrically with $\epsilon$, second, the condition in the theorem that $|\epsilon| \leq n^{\alpha -1}$, and third, the expansion $e^x = 1 + x + \frac{x^2}{2}+\dots$

Again, my specific issue is in making the jump from (1.87) to (1.88).

My Attempt

I have tried many routes, the most promising of which utilized the property of a geometric series: $$a + ar + ar^2 + \dots = \frac{a}{1 - r}$$

Which, applied to (1.87), seems to show that it is decreasing geometrically at least with $\epsilon$, and has an initial term of $\epsilon^3(\frac{1}{6p^2} - \frac{1}{6q^2})$:

$$r = \epsilon$$ $$a = \epsilon^3(\frac{1}{6p^2} - \frac{1}{6q^2})$$

And hence:

$$\frac{a}{1 - r} = \frac{\epsilon^3(\frac{1}{6p^2} - \frac{1}{6q^2})}{1 - \epsilon}$$

Plugging in $n^{\alpha -1}$ for $\epsilon$ I get:

$$ \frac{(n^{\alpha -1})^3(\frac{1}{6p^2} - \frac{1}{6q^2})}{1 - n^{\alpha -1}}$$

And then letting $C_2 = \frac{1}{6p^2} - \frac{1}{6q^2}$, I arrive at:

$$ \frac{(n^{\alpha -1})^3 C_2}{1 - n^{\alpha -1}}$$

But I am not able to make the final jump to (1.88), which leads me to believe I have made a mistake somewhere in this reasoning.

Any insight into how to go from (1.87) to (1.88) is much appreciated!

Answer 1

Your solution is correct. A premise of the theorem is that $\alpha$ is a fixed constant such that $1/2 < \alpha < 2/3$. For such $\alpha$ and large $n$ (as required by the theorem), the expression

$$ \frac{c\cdot n^{3\alpha - 2}}{1 - n^{\alpha - 1}} $$

is a very small number, because both terms $n^{3\alpha-2}$ and $n^{\alpha-1}$ go to zero for large $n$.

Then, as suggested by Gallager, we can use the expansion for $e^x$ for small x.

$$ \exp\left(-\frac{c\cdot n^{3\alpha - 2}}{1 - n^{\alpha - 1}}\right) \approx 1 - \frac{c\cdot n^{3\alpha - 2}}{1 - n^{\alpha - 1}} \approx 1-c\cdot n^{3\alpha-2} $$

At this point, I believe there are typos in the proof because if $\alpha < 2/3$, terms such as $n^{-3\alpha + 2}$ will blow up for large $n$. Also, the original statement of the theorem uses $n^{3\alpha - 2}$.

Answer 2

This is not an answer but rather a long comment.

For this kind of problems which require composition of series, I think that it easier to start from where we want to end (!!). For your case $$A=e^{-n S} \implies \log(A)=-n S \tag 1$$ $$S=(p+\epsilon ) \log \left(1+\frac{\epsilon }{p}\right)+(q-\epsilon ) \log \left(1-\frac{\epsilon }{q}\right)$$ $$S=\frac{ (p+q)}{2 p q}\epsilon ^2+\frac{1}{6} \left(\frac{1}{q^2}-\frac{1}{p^2}\right)\epsilon ^3+O\left(\epsilon ^4\right)$$ Now, continuing with Taylor series using $A=e^{\log(A)}$ you will end with $$A=1-\frac{ n (p+q)}{2 p q}\epsilon ^2+\frac{1}{6} n \left(\frac{1}{p^2}-\frac{1}{q^2}\right)\epsilon ^3+O\left(\epsilon ^4\right)$$ which simplify if $p+q=1$ and then write

$$A=1-\frac{ n }{2 p q}\epsilon ^2+\frac{ n(q-p)}{6p^2q^2}\epsilon ^3+O\left(\epsilon ^4\right)$$