Posterior Distribution of a Prior Variable

Question

Let $X_{1},\dots,X_{n}$ be a random sample from an exponential distribution with density $f(x;\theta)=\theta e^{-\theta x}$, $x>0$ (having mean $1/\theta$). Assume a prior density for $\theta$ which is also exponential with mean $1/\beta$, where $\beta$ is known. Prove that the posterior distribution of $\beta$ is a gamma distribution.

--In general, the posterior distribution would be $\pi(\beta\mid \mathbf{x})=f(\mathbf{x}\mid \beta)\pi(\beta)/m(\mathbf{x})$, where $m(\mathbf{x})$ is the marginal probability of $x$. The part that I am stuck at has to do with being given $f(\mathbf{x}\mid\theta)$, not $f(\mathbf{x}\mid\beta)$. Any help is appreciated!

Answer 1

The likelihood function is $$ L(\theta) = \prod_{i=1}^n \left(\theta e^{-\theta x_i} \right) = \theta^n e^{-\theta(x_1+\cdots+x_n)}\text{ for }\theta>0. $$

The prior density is $$ f(\theta) = \frac1\beta e^{-\theta/\beta}\text{ for }\theta>0. $$

Multiply the two: $$ \text{constant}\cdot \theta^n e^{-\theta\left(x_1+\cdots+x_n+\frac1\beta\right)}, $$ where in this case "constant" means not depending on $\theta$.

Suitably normalized, that is a gamma density. The normalizing constant will generally differ from the "constant" above. You can find the normalizing constant just by looking at tabulated facts about the gamma density.

Possibly you don't even need the normalizing constant. If one parametrizes the family of gamma densities as $$ f(t) = \text{constant}\cdot t^{\alpha-1} e^{-\gamma t}\text{ for }t>0, $$ then in this case we have $\alpha = n+1$ and $\gamma=x_1+\cdots+x_n+\frac1\beta$.

Answer 2

By Bayes theorem, the posterior distribution over $\theta$ is given by $$ P(\theta\mid x_1,\ldots,x_n) = \frac{P(x_1,\ldots,x_n\mid\theta)P(\theta)}{P(x_1,\ldots,x_n)}. $$ Now, since $x_i$ are all independent, we have $$ P(x_1,\ldots,x_n\mid\theta) = P(x_1\mid\theta)\cdots P(x_n\mid\theta) = \theta^n e^{-\theta\sum_i x_i}. $$ Also, the prior is $P(\theta) = \beta e^{-\beta\theta}$. Because $P(x_1,\ldots,x_n)$ is not a function of $\theta$, we can write $$ P(\theta\mid x_1,\ldots,x_n) = \text{const}\times\theta^n e^{-\theta\sum_i x_i-\theta\beta}, $$ where the constant of proportionality will be fixed by requiring that $\int P(\theta\mid x_1,\ldots,x_n)\,d\theta=1$.