Laplace approximation of Poisson posterior from MacKay

Question

I am doing exercise 27.1 on Laplace's method from David MacKay's textbook, which is to make a Laplace approximation of a Poisson model with an improper prior:

$$ p(x \mid \lambda) = \frac{e^{-\lambda} \lambda^x}{x!}, \quad p(\lambda) = \frac{1}{\lambda} $$

The posterior is

$$ p(\lambda \mid x) = \frac{p(x \mid \lambda) p(\lambda)}{Z} = \frac{\frac{e^{-\lambda} \lambda^{x-1}}{x!}}{Z} $$

where $Z$ is the unknown normalizer. My understanding is that we want a Laplace approximation of

$$ f(x) = \frac{e^{-\lambda} \lambda^{x-1}}{x!} $$

And this requires taking the second derivative of the log of this function:

\begin{align} \frac{d}{dx} \log f(x) &= \frac{d}{dx} \Big( -\lambda + (x - 1) \log \lambda - \log x!\Big) \\\\ &= \frac{d}{dx} \Big( -\lambda + x \log \lambda - \log \lambda - \log x!\Big) \\\\ &= \log \lambda - \color{red}{\frac{d}{dx} \log x!} \end{align}

I am stuck on the $\color{red}{\text{red}}$ term above. Here is what I have tried:

\begin{align} \require{cancel} \frac{d}{dx} \log x! &= \frac{d}{dx} \log \Gamma(x+1) \\\\ &= \frac{\Gamma^{\prime}(x+1)}{\Gamma(x+1)} \\\\ &= \frac{\cancel{\Gamma(x+1)} \psi_0(x+1)}{\cancel{\Gamma(x+1)}} \\\\ &= \psi_0(x+1) \end{align}

where $\psi_0$ is the polygamma function. But now I don't know how to compute this derivative.

Answer 1

The answer here is that we can ignore $\log (x!)$ because it does not change the estimate of $\lambda$.

Answer 2

According to the wikipedia page, the derivative of $\psi_0(x+1)$ is $\psi^{(1)}(x+1)$.

Its series representation can be written as $$\sum_{k=1}^\infty\frac1{(x+k)^2}=\zeta(2,x+1).$$

Answer 3

The posterior is a function of $\lambda$ so you need to take 1st and 2nd derivatives of the log-posterior wrt to $\lambda$. At that point the factorial $x!$ disappears and so do all terms not depending on $\lambda$ as @gwg mentioned.