Solving for a Conjugate Prior in search of MAP estimator

59 Views Asked by At

I am trying to prove that if a given random variable $X \sim Exp(\lambda)$ and $\lambda \sim Gamma(\alpha,\beta)$ hen $\lambda | X \sim Gamma(\alpha^{*},\beta^{*})$ for some parameters $\alpha^{*}$ and $\beta^{*}$. Here is a rough outline of what I have so far:

I get that this is a Baye's rule proportionality problem, and we see $$P(\lambda | X) \propto P(X | \lambda) P(\lambda) = \left(\prod_{i=1}^n P(X_i | \lambda)\right) P(\lambda) = \left(\prod_{i=1}^n P(X_i | \lambda)\right)\frac{\beta^{\alpha}}{ \Gamma(\alpha)} \lambda^{\alpha-1} e^{-\beta \lambda}$$ $$=\left(\prod_{i=1}^n \lambda e^{-\lambda X_i} \right)\frac{\beta^{\alpha}}{ \Gamma(\alpha)} \lambda^{\alpha-1} e^{-\beta \lambda}.$$ However, I am having some trouble moving forward with this problem, specifically how we adjust $\beta^{\alpha}$ to get a gamma distribution. any suggestions would be greatly appreciated.

1

There are 1 best solutions below

0
On

First, observe that $$\prod_{i=1}^n \lambda e^{-\lambda x_i} = \lambda^n e^{-\lambda \sum x_i} = \lambda^n e^{-n \bar x \lambda},$$ where $\bar x = \frac{1}{n} \sum_{i=1}^n x_i$ is the sample mean. Then remove every factor that is not a function of $\lambda$: $$f(\lambda \mid \boldsymbol x) \propto \lambda^n e^{-n \bar x \lambda} \lambda^{\alpha - 1} e^{-\beta \lambda}.$$ This is because the terms $\beta^\alpha$ and $\Gamma(\alpha)$ are constants with respect to the sample and with respect to the parameter $\lambda$. Then simplify: $$f(\lambda \mid \boldsymbol x) \propto \lambda^{n + \alpha - 1} e^{-(\beta + n \bar x) \lambda}.$$ Again, with respect to $\lambda$, this is proportional to a gamma distribution with posterior hyperparameters $\alpha^* = n + \alpha$ and $\beta^* = \beta + n \bar x.$