Derivation of a conditional probability mass function which involves Geometric and Gamma random variables

Question

In the $6$th chapter of the textbook A first course in probability by Ross, there is the following question:

Let $N$ be a geometric random variable with parameter $p.$ Suppose that the conditional distribution of $X$ given that $N = n$ is the gamma distribution with parameters $n$ and $\lambda.$ Find the conditional probability mass function of $N$ given that $X = x.$

At first, I'll setup notation and a solution. Then, I'll start towards another solution, which I cannot complete at the moment.

We are given $f_{X|N=n}(x) = \frac{\lambda e^{- \lambda x} (\lambda x)^{n-1}}{\Gamma(n)} = \frac{\lambda e^{- \lambda x} (\lambda x)^{n-1}}{(n-1)!},$ since $n$ is a positive integer; and we want to find $P(N=n | X=x).$ So, we write $$P(N=n | X=x) = \frac{f_{X|N=n}(x) P(N=n)}{f_{X}(x)},$$ which is equal to $$c \frac{(\lambda(1-p)x)^{n-1}}{(n-1)!},$$ where $c$ is a term that does not depend upon $n.$ Now, we know that $$1 = c \sum_{n=1}^{\infty} \frac{(\lambda(1-p)x)^{n-1}}{(n-1)!} = c e^{\lambda(1-p)x},$$ so $c$ must be equal to $e^{- \lambda(1-p)x}.$ Thus, $$P(N=n | X=x) = e^{- \lambda(1-p)x} \frac{(\lambda(1-p)x)^{n-1}}{(n-1)!}$$ is equivalent to a Poisson probability mass function, where the parameter is $\lambda(1-p)x$ and $n=1$ corresponds to $1 - 1 = 0,$ and so on, in terms of the numerical values.

Now, the alternative solution starts the same, but aims to avoid the constant term $c.$ That is, $$P(N=n | X=x) = \frac{f_{X|N=n}(x) P(N=n)}{f_{X}(x)} = \frac{\frac{\lambda e^{- \lambda x} (\lambda x)^{n-1}}{(n-1)!} (1-p)^{n-1}p}{\sum_{n=1}^{\infty} \frac{\lambda e^{- \lambda x} (\lambda x)^{n-1}}{(n-1)!}},$$ which simplifies to $$\bigg(\frac{p}{\sum_{n=1}^{\infty} \frac{(\lambda x)^{n-1}}{(n-1)!}} \bigg) \bigg(\frac{(\lambda(1-p)x)^{n-1}}{(n-1)!}\bigg).$$

So, the first term in parenthesis is equivalent to $c.$ Moreover, it is equal to $$\frac{p}{\sum_{m=0}^{\infty} \frac{(\lambda x)^{m}}{(m)!}} = \frac{p}{e^{\lambda x}} = p e^{- \lambda x}.$$ But I do not see how to proceed from here, or, where I should trace back to in the derivation and proceed from there.

Answer 1

The infinite series in the denominator is missing a factor of $\left(1-p\right)^{n-1}p$. Basically you need to add up the value from the numerator over all possible $n$. After including this factor, you can cancel out the $p$ in the numerator, and the series will sum to $e^{\lambda\left(1-p\right)x}$, which is what you need.

Answer 2

It is much more convenient to work with the kernels of each distribution, rather than the density and PMF.

Our hierarchical model is $$N \sim \operatorname{Geometric}(p), \\ X \mid N \sim \operatorname{Gamma}(N, \lambda), \\ \Pr[N = n] = (1-p)^{n-1} p, \quad n \in \{1, 2, \ldots \}, \\ f_{X \mid N}(x) = \frac{\lambda^N x^{N-1} e^{-\lambda x}}{\Gamma(N)}, \quad x > 0.$$

Then as you noted, Bayes' rule gives

$$\Pr[N = n \mid X = x] = \frac{f_{X \mid N}(x) \Pr[N = n]}{\Pr[X = x]}.$$ But we don't care about the denominator $\Pr[X = x]$, because it does not depend on $n$. All that is needed is to observe that the kernel of the LHS must equal the product of the kernels on the RHS, up to some constant of proportionality with respect to $n$:

$$\Pr[N = n \mid X = x] \propto \frac{\lambda^n x^{n-1}}{(n-1)!} (1-p)^{n-1} \propto \frac{(\lambda x (1-p))^{n-1}}{(n-1)!},$$ where we have removed every factor that does not depend on $n$. In the second step, we removed a constant factor of $\lambda$. This gives us the kernel of a Poisson distribution with rate $\lambda^* = \lambda x (1-p)$; that is to say, the posterior distribution of $N - 1$ given the observation $X = x$ is Poisson with rate $\lambda^*$.

The location transformation is required because we selected a parametrization of the prior geometric distribution of $N$ that had support on the strictly positive integers, hence the posterior cannot have $N = 0$. Thus $N - 1 \mid X$ is Poisson, rather than $N \mid X$.