A result regarding Hierarchical Bayes

33 Views Asked by At

I have the following,

$$x_i \mid \theta_i \sim \text{Bin}(m, \theta_i), ~i=1,\dots,n,$$ $$\theta_i \sim \text{Beta}(\alpha,1),$$ $$ f(\alpha) \propto 1.$$

I wish to compute $f(\theta_1,\dots,\theta_n,\alpha\mid x_1,\dots,x_n)$.

By Bayes' Theorem,

$$f(\theta_1,\dots,\theta_n,\alpha\mid x_1,\dots,x_n) \propto f(x_1,\dots,x_n \mid \theta_1,\dots,\theta_n,\alpha)f(\theta_1,\dots,\theta_n,\alpha)$$

$$\propto f(x_1,\dots,x_n \mid \theta_1,\dots,\theta_n,\alpha)f(\theta_1,\dots,\theta_n\mid\alpha)f(\alpha)$$

From which I get,

$$f(\theta_1,\dots,\theta_n,\alpha\mid x_1,\dots,x_n) \propto \theta_1^{\alpha+x_1 - 1}(1 - \theta_1)^{m - x_1}\dots\theta_n^{\alpha+x_n - 1}(1 - \theta_n)^{m - x_n}$$

However, in my notes this expression has an $\alpha^n$ at the start which I am not sure about.

Any help would be appreciated, thanks.

1

There are 1 best solutions below

0
On BEST ANSWER

I imagine your samples and priors are independent. I will denote posteriors by $p$, priors by $\pi$, likelihoods by $f$ and data by $\mathcal{D}:=\{x_1,\dots,x_n\}$.

Observe that you wish to compute the posterior density of $\theta_1,\dots,\theta_n,\alpha$ so you have to be careful in order to not throw away dependencies on these parameters.

Your formulas are correct: since the samples and $\theta_i$'s are independent we have \begin{align*} p(\theta_1,\dots,\theta_n,\alpha| \mathcal{D}) \propto& f(\mathcal{D} |\theta_1,\dots, \theta_n,\alpha) \pi(\theta_1,\dots, \theta_n,\alpha)\\ =& f(\mathcal{D} |\theta_1,\dots, \theta_n,\alpha) \pi(\theta_1,\dots, \theta_n|\alpha)\pi(\alpha)\\ =& f(\mathcal{D} |\theta_1,\dots, \theta_n,\alpha) \pi(\theta_1|\alpha) \cdots \pi(\theta_n|\alpha) \pi(\alpha) \end{align*} where here $B(\alpha,1)$ denotes the beta function and comes from the expression for the density of a beta distribution.

We further have \begin{align*} p(\theta_1,\dots,\theta_n,\alpha| \mathcal{D}) \propto& \prod_{i=1}^n \left[\begin{pmatrix} m \\ x_i\end{pmatrix} \theta_i^{x_i} (1-\theta_i)^{m-x_i} \frac{1}{B(\alpha,1)}\theta_i^{\alpha-1}\right] \end{align*} Now, observe that $$\frac{1}{B(\alpha,1)}=\frac{\Gamma \left( \alpha + 1 \right)}{\Gamma \left( \alpha \right) \Gamma\left( 1 \right)} =\frac{\alpha!}{(\alpha-1)!}= \alpha.$$ Hence, \begin{align*} p(\theta_1,\dots,\theta_n,\alpha| \mathcal{D}) \propto & \, \alpha^n \prod_{i=1}^n \left[\theta_i^{x_i+\alpha-1} (1-\theta_i)^{m-x_i} \right] \end{align*} which looks almost like what you did. So I imagine your only mistake was to forget the term $\frac{1}{B(\alpha,1)}$ from the beta distribution of the $\theta_i$'s.

I hope this was helpful. Good luck!