How to rigorously understand the idea behind Laplace method?

Question

Let us consider the integral $$I(\lambda)=\int_a^b f(x) e^{-\lambda g(x)} dx\tag{1}$$ where $f$ and $g$ are smooth functions. The Laplace method (sketched, e.g. here) allows us to take $\lambda \to \infty$ in this expression. The idea seems to be that in the limit $\lambda\to \infty$ only neighborhoods of minima of $g$ will contribute. This is the idea I want to understand from a rigorous point of view.

In other words, suppose that $g$ has just one minimum $c\in[a,b]$ with $g'(c)=0$ and $g''(c)>0$. In that case one approximates $$I(\lambda)\approx \int_{c-\epsilon}^{c+\epsilon} f(x) e^{-\lambda g(x)}dx\tag{2}$$

and since this is one small interval around $c$ one Taylor expands $f$ and $g$.

Now I want to rigorously understand (2). I mean, one is clearly separating $$I(\lambda)=\int_{(a,b)\setminus (c-\epsilon,c+\epsilon)}f(x)e^{-\lambda g(x)}dx+\int_{c-\epsilon}^{c+\epsilon}f(x)e^{-\lambda g(x)}dx\tag{3}$$ and saying that we can throw the first term away.

But how to understand the accuracy of the approximation with respect to $\epsilon$? Moreover, why the $\lambda \to +\infty$ is needed to do this, and how this limit relates to the choice of $\epsilon$?

In other words what is the rigorous way to make sense of the asymptotic approximation (2)?

Answer 1

• $\lambda \rightarrow \infty $. That is how Laplace Transform is defined. The Laplace Transform of $f\left ( t \right )$, denoted by $\textit{F}\left ( s \right )=\mathfrak{L}\left \{ f\left ( t \right ) \right \}$, is defined by the integral $F\left ( s \right )=\int_{0}^{\infty }e^{-st}f\left ( t \right )dt$, where $f\left ( t \right )$ is defined for $t\geq 0$. This unilateral Laplace Transform is what is meant by "the" Laplace Transform most of the time, although sometimes the bilateral form is also used, $F\left ( s \right )=\int_{-\infty }^{\infty }e^{-st}f\left ( t \right )dt$.

• $I\left ( \lambda \right )=\int_{a}^{b}f\left ( t \right )e^{-\lambda g\left ( t \right )}dt\approx e^{-\lambda g\left ( c \right )}\int_{c-\epsilon }^{c+\epsilon }f\left ( t \right )e^{-\lambda \left [ g\left ( t \right )-g\left ( c \right ) \right ]}dt$ for large $\lambda$.

If the real continuous function $g\left ( t \right )$ assumes strict minimum on the interval $\left [ a,b \right ]$ at $c \in \left [ a,b \right ]$, then it is only the immediate neighborhood of $c$ that contributes to the asymptotic expansion of $I\left ( \lambda \right )$ for large $\lambda$. Furthermore, as $g\left ( t \right )$ assumes its strict minimum on the interval, $-g\left ( t \right )$ assumes its strict maximum on the interval.

$I\left ( \lambda \right )=\int_{a}^{b}f\left ( t \right )e^{-\lambda g\left ( t \right )}dt=\int_{a}^{b}f\left ( t \right )e^{\left(\phi \left ( t \right )\right)\lambda }dt$ where $\phi \left ( t \right )=-g\left ( t \right )$

Write $e^{\phi \left ( t \right )}$ as $e^{\phi \left ( c \right )}e^{\psi \left ( t \right )}$, where $e^{\phi \left ( c \right )}$ is a multiplication constant that can be taken out of the integral. Immediately, it follows that $\psi \left ( t \right )=\phi \left ( t \right )-\phi \left ( c \right )$. The maximum value of $ \psi \left ( t \right )$ is $0$, that is because the largest $\phi \left ( t \right )$ can be is $\phi \left ( c \right )$. The maximum value for $e^{\psi \left ( t \right )}$ is therefore $1$.

Following is a sketch for the typical behavior of $e^{\psi \left ( t \right )}$. Recall that $$\left.\begin{matrix}\phi \left ( t \right )=-g\left ( t \right )\\ \psi \left ( t \right )=\phi \left ( t \right )-\phi \left ( c \right )\end{matrix}\right\}\rightarrow \psi \left ( t \right )=-g\left ( t \right )-\left ( -g\left ( c \right ) \right )=-g\left ( t \right )+g\left ( c \right )\\ \psi \left ( t \right )=-\left [ g\left ( t \right )-g\left ( c \right ) \right ]$$ Hence, this graph is also the typical behavior for $e^{-\left [ g\left ( t \right )-g\left ( c \right ) \right ]}$

As $\lambda$ increases, the maximum for $e^{\psi \left ( t \right )}$ stays at $\left ( c,1 \right )$. However, the wings of the graphs move closer and closer to the vertical line $t=c$.Thus, $\phi \left ( t \right )$ (or $-g\left ( t \right )$) can be replaced with its approximation that only needs to be good in the immediate vicinity of $c$, where $\phi \left ( t \right )$ assumes it maximum (or equivalently, where $g\left ( t \right )$ assumes its minimum.)

Put everything together, we therefore have Expression (2) from the lecture notes you included in your question.

$$I\left ( \lambda \right )=\int_{a}^{b}f\left ( t \right )e^{\left ( \phi \left ( t \right ) \right )\lambda }\approx e^{\phi \left ( c \right )}\int_{c-\epsilon }^{c+\epsilon }f\left ( t \right )e^{\left ( \phi \left ( t \right ) \right )\lambda }dt=e^{-\lambda g\left ( c \right )}\int_{c-\epsilon }^{c+\epsilon }f\left ( t \right )e^{-\lambda \left [ g\left ( t \right )-g\left ( c \right ) \right ]}dt$$

Answer 2

I will work with the case where $f(x)$ is a constant function. It provides all the essentials, the general case is just a matter of additional details.

In your Eq. (3), the second integral can be estimated in the following manner: $$ \int_{c-\epsilon}^{c+\epsilon}e^{-\lambda g(x)} dx = \int_{c-\epsilon}^{c+\epsilon}e^{-\lambda \left( g(c) + g''(x_*) \frac{(x-c)^2}{2} \right)} dx = \int_{-\epsilon}^{\epsilon}e^{-\lambda \left( g(c) + g''(x_*) \frac{y^2}{2} \right)} dy $$

where $x_*$ is some number between $c$ and $x$, dependent on $x$ (and hence $y$).

Now consider $\epsilon$ small enough such that $g''(x)$ is positive on the whole of $[c-\epsilon, c+\epsilon]$. Certainly such an $\epsilon$ exists since $g''$ is assumed to be continuously twice differentiable. For the same reason, it takes a minimum and a maximum on $[c-\epsilon, c+\epsilon]$, let us call them $g''_{m}$ and $g''_{M}$, respectively. Thus we have:

$$ \int_{-\epsilon}^{\epsilon}e^{-\lambda g''_M \frac{y^2}{2} } dy \leq \int_{-\epsilon}^{\epsilon}e^{-\lambda g''(x_*) \frac{y^2}{2} } dy \leq \int_{-\epsilon}^{\epsilon}e^{-\lambda g''_m \frac{y^2}{2} } dy $$

Let $z=y/{\epsilon}$, then

$$ \epsilon \int_{-1}^{1}e^{-\lambda g''_M \frac{\epsilon^2 z^2}{2} } dz \leq \text{ Main Guy} \leq \epsilon \int_{-1}^{1}e^{-\lambda g''_m \frac{z^2}{2} } dz $$

$\implies$ $$ \epsilon \int_{-\infty}^{\infty}e^{-\lambda (g''(c)+\delta(\epsilon)) \frac{\epsilon^2 z^2}{2} } dz \leq \text{ MG} + \epsilon \int_{(-\infty,\infty)\setminus [-1,1]}e^{-\lambda g''(c)\frac{\epsilon^2 z^2}{2} }dz \leq \epsilon \int_{-\infty}^{\infty}e^{-\lambda (g''(c)-\delta(\epsilon)) \frac{z^2}{2} } dz $$ for appropriate $\delta$ (i.e., the maximum of $g''(c)-g''_m$ and $g''_M-g''(c)$). This gives

$$ \frac{1}{\sqrt{\lambda}} \sqrt{\frac{2\pi}{g''(c)+\delta(\epsilon)}} \leq \text{ MG} + \epsilon \int_{(-\infty,\infty)\setminus [-1,1]}e^{-\lambda g''(c)\frac{\epsilon^2 z^2}{2} }dz \leq \frac{1}{\sqrt{\lambda}} \sqrt{\frac{2\pi}{g''(c)-\delta(\epsilon)}} $$

$\implies$ $$ 1 - \frac{1}{2} \frac{\delta(\epsilon_*)}{g''(c)} \leq \frac{\sqrt{\lambda}}{\sqrt{2\pi/g''(c)}} \text{ MG} + \frac{\sqrt{\lambda}}{\sqrt{2\pi/g''(c)}} \epsilon \int_{(-\infty,\infty)\setminus [-1,1]}e^{-\lambda g''(c)\frac{\epsilon^2 z^2}{2} }dz \leq 1 + \frac{1}{2} \frac{\delta(\epsilon_{**})}{g''(c)} $$ for some $\epsilon_*$, $\epsilon_{**}$ lying between $0$ and $\epsilon$.

Now let us closely look at the various terms in this inequality. Particularly, the second integral in between the two less-than-equal signs can be estimated as: $$ 0 \leq \int_{(-\infty,\infty)\setminus [-1,1]}e^{-\lambda g''(c)\frac{\epsilon^2 z^2}{2} }dz = 2 \int_{1}^{\infty}e^{-\lambda g''(c)\frac{\epsilon^2 z^2}{2} }dz \leq 2 \int_{1}^{\infty} z e^{-\lambda g''(c)\frac{\epsilon^2 z^2}{2} }dz = \frac{2 e^{-\lambda g''(c)\frac{\epsilon^2}{2} } }{\lambda g''(c) \epsilon^2} $$

Therefore, $$ - \frac{1}{2} \frac{\delta_*(\epsilon)}{g''(c)} - \frac{2 e^{-\lambda g''(c)\frac{\epsilon^2}{2} } }{\epsilon \sqrt{2 \pi \lambda g''(c)} } \leq \frac{\sqrt{\lambda}}{\sqrt{2\pi/g''(c)}} \text{ MG} -1 \leq + \frac{1}{2} \frac{\delta_{*}(\epsilon)}{g''(c)} + \frac{2 e^{-\lambda g''(c)\frac{\epsilon^2}{2} } }{\epsilon \sqrt{ 2 \pi \lambda g''(c)}} $$

i.e., $$ \left| \frac{\sqrt{\lambda}}{\sqrt{2\pi/g''(c)}} \text{ MG} -1 \right| \leq \frac{1}{2} \frac{\delta_{*}(\epsilon)}{g''(c)} + \frac{2 e^{-\lambda g''(c)\frac{\epsilon^2}{2} } }{\epsilon \sqrt{ 2 \pi \lambda g''(c)}} \tag{H1}$$ where, $\delta_*(\epsilon)$ is the larger of $\delta(\epsilon_*)$ and $\delta(\epsilon_{**})$. Note that, so far, the largeness of $\lambda$ hasn't started making an appearance: Eq. (H1) is true for all positive $\lambda$, just that $\epsilon$ has to be appropriately small (need not be vanishingly, at least not yet, we will get there soon).

Now let us estimate the other integral of Eq. (3) in the original post: $$ \text{Other Guy}:= \int_{(a,b)\setminus (c-\epsilon,c+\epsilon)} e^{-\lambda g(x)}dx \leq \int_{(a,b)\setminus (c-\epsilon,c+\epsilon)} e^{-\lambda g_m(\epsilon)}dx \leq (b-a) e^{-\lambda g_m(\epsilon)} $$ where $g_m$ is the minimum of $g(x)$ on $(a,b) \setminus (c-\epsilon, c+\epsilon)$, and therefore, by the given data, greater than $g(c)$.

Now let us club this inequality with Eq. (H1) to give $$ \left| \frac{\sqrt{\lambda}}{\sqrt{2\pi/g''(c)}} \text{ MG} + \frac{ e^{\lambda g(c)}\sqrt{\lambda}}{\sqrt{2\pi/g''(c)}} \text{OG} -1 \right| \leq \frac{1}{2} \frac{\delta_{*}(\epsilon)}{g''(c)} + \frac{2 e^{-\lambda g''(c)\frac{\epsilon^2}{2} } }{\epsilon \sqrt{ 2 \pi \lambda g''(c)}} + \frac{(b-a) \sqrt{\lambda}}{\sqrt{2\pi/g''(c)}} e^{-\lambda \left( g_m(\epsilon) - g(c) \right)} $$

$\implies$ $$ \left| \frac{ e^{\lambda g(c)}\sqrt{\lambda g''(c) } }{\sqrt{2\pi}} I(\lambda) -1 \right| \leq \frac{1}{2} \frac{\delta_{*}(\epsilon)}{g''(c)} + \frac{2 e^{-\lambda g''(c)\frac{\epsilon^2}{2} } }{\epsilon \sqrt{ 2 \pi \lambda g''(c)}} + \frac{(b-a) \sqrt{\lambda}}{\sqrt{2\pi/g''(c)}} e^{-\lambda \left( g_m(\epsilon) - g(c) \right)} \tag{H2} $$ where $I(\lambda)$ is the our main integral of interest from the original post. Note that while MG and OG are dependent on $\epsilon$, the term on the left in the preceding inequality has no such dependence.

Now we are in a position to answer all the questions asked all at once. We use the ''epsilon-delta'' definition to prove that the term in the l.h.s of Eq. (H2) vanishes in the limit of $\lambda$ tending to $\infty$, thereby establishing rigorously the asymptotic behavior of $I(\lambda)$: Given any $\xi > 0$, choose an $\epsilon$ small enough such that the first term on the r.h.s of Eq. (H2) is less than $\xi/3$. Next, for that $\epsilon$, choose a $\lambda$ such that the second term is less than $\xi/3$. This can always be done by choosing big enough $\lambda$. If this value is sufficient to make the last term on the r.h.s of Eq. (H2) also to be less than $\xi/3$, we are done, if not, certainly there will be a bigger one which will be sufficient. This concludes the demonstration.

All the questions asked are essentially answered. Particularly, the question of why Eq. (2) is true for all positive $\epsilon$, can now be answered by noting that $\frac{\sqrt{\lambda g''(c)}}{\sqrt{2 \pi}}$MG($\epsilon$) can be written as $$ \frac{\sqrt{\lambda g''(c)}}{\sqrt{2 \pi}} \text{ MG}(\epsilon) = \frac{ e^{\lambda g(c)}\sqrt{\lambda g''(c) } }{\sqrt{2\pi}} I(\lambda) - \frac{ e^{\lambda g(c)}\sqrt{\lambda g''(c) } }{\sqrt{2\pi}} \text{ OG}(\epsilon), $$ and both the terms in the r.h.s have limits as $\lambda \to \infty$. One of them is zero and the other is unity. Therefore by limit laws, we have Eq. (2) of the original post.