A question about posterior density

Question

This exercise is the first question from the book: Bayesian Data Analysis.

Question:Conditional probability: suppose that if θ=1, then y has a normal distribution with mean 1 and standard deviation σ, and if θ=2, then y has a normal distribution with mean 2 and standard deviation σ. Also, suppose Pr(θ=1)=0.5 and Pr(θ=2)=0.5.

Describe how the posterior density of θ changes in shape as σ is increased and as it is decreased.

I know the posterior density is calculated in this way:

The solution is: As σ → ∞, the posterior density for θ approaches the prior (the data contain no information): Pr(θ = 1|y = 1) → 1/2=0.5 . As σ → 0, the posterior density for θ becomes concentrated at 1: Pr(θ = 1|y = 1) → 1.

But how can I get this solution in a precise way? Currently the solution is in an intuitive way, which is helpful but not precise.

Answer 1

Let $\phi(x;m,\sigma):=\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\tfrac{(x-m)^2}{2\sigma^2}}$. We arę given $$f(x|\theta)=\phi(x;1,\sigma)\mathbb{1}_{\{1\}}(\theta)+\phi(x;2,\sigma)\mathbb{1}_{\{2\}}(\theta)$$ and a prior disttribution $$\pi(\theta)=\frac{1}{2}\Big(\mathbb{1}_{\{1\}}(\theta)+\mathbb{1}_{\{2\}}(\theta)\Big)$$ The posterior distribution is given by $$\pi(\theta|x)=\frac{f(x|\theta)}{m(x)}\pi(\theta)$$ where $m(x)=\int_\Theta f(x|\theta)\,\pi(d\theta)$, the marginal of $x$, can be considered as a normalizing factor. In our case, \begin{align}\pi_\sigma(\theta|x)&=\frac{\phi(x;1,\sigma)\mathbb{1}_{\{1\}}(\theta) + \phi(x;2,\sigma)\mathbb{1}_{\{2\}}(\theta)}{\phi(x;1,\sigma) + \phi(x;2,\sigma)}\\ &=\frac{e^{-\tfrac{(x-1)^2}{2\sigma^2}}\mathbb{1}_{\{1\}}(\theta) + e^{-\tfrac{(x-2)^2}{2\sigma^2}}\mathbb{1}_{\{2\}}(\theta)}{e^{-\tfrac{(x-1)^2}{2\sigma^2}} + e^{-\tfrac{(x-2)^2}{2\sigma^2}}} \tag{1}\label{one}\end{align}

• As $\sigma\rightarrow0$, $\exp\big(-\frac{(x-m)^2}{2\sigma^2}\big)\rightarrow1$ whence we get that $$\begin{align}\lim_{\sigma\rightarrow\infty}\pi_\sigma(\theta|x)=\frac12\big(\mathbb{1}_{\{1\}}(\theta)+\mathbb{1}_{\{2\}}(\theta)\Big)=\pi(\theta)\end{align}$$

• For the case $\sigma\rightarrow\infty$ we will use the following observation:

Let $a,b>0$. Then, $$\lim_{t\rightarrow\infty}\frac{e^{-at}}{e^{-bt}+e^{-at}}=\frac12\mathbb{1}(a=b)+\mathbb{1}(a<b)$$

Applying this to \eqref{one} yields $$\lim_{\sigma\rightarrow0}\pi_\sigma(\theta|x)=\left\{ \begin{matrix}\mathbb{1}_{\{1\}}(\theta) &\text{if} & x<3/2\\ \frac12\big(\mathbb{1}_{\{1\}}(\theta)+\mathbb{1}_{\{2\}}(\theta)\big) & \text{if} & x=\frac32\\ \mathbb{1}_{\{2\}}(\theta) &\text{if} & x>3/2 \end{matrix} \right. $$