Why is this posterior distribution not well defined when $X=0$?

145 Views Asked by At

I just learned about improper prior distributions:

Def: A prior distribution with density $f_\Theta$ satisfying $$\int f_\Theta(\theta) d\theta = \infty$$

is called an improper prior distribution.

The fact that improper priors can only be used as long as the posterior is proper, is illustrated by the following example:

Example:

Suppose $X\sim\operatorname{Bin}(n,\theta)$ and we use the improper prior $f_\Theta(\theta) = \dfrac{1}{\theta(1-\theta)}$. It follows that $$f_{\Theta|X}(\theta|x)\propto\dfrac{1}{\theta(1-\theta)}\theta^x(1-\theta)^{n-x} = \theta^{x-1}(1-\theta)^{n-x-1}.$$ Clearly, $\theta\mapsto f_{\Theta|X}(\theta|0)$ is not integrable. Hence, there is no well-defined posterior when $X=0$.

Question: Why is it so clear that $\theta\mapsto f_{\Theta|X}(\theta|0)$ is not integrable, and why is the notation $\mapsto$ used here? Furthermore, why is $\displaystyle\int\dfrac{1}{\theta(1-\theta)}d\theta$ improper? $\displaystyle\int\dfrac{1}{\theta(1-\theta)}d\theta = \log(\theta) - \log(1-\theta)+c$, and $\theta\in[0,1]$, so why would this be equal to $\infty$?

Thanks in advance!

1

There are 1 best solutions below

0
On BEST ANSWER

When $X=0$, we have $f_{\Theta|X}(\theta|0)=\theta^{-1}(1-\theta)^{n-1}$.

An integral of the form $\int_0^1 t^a (1-t)^b dt$ does not converge if either $a$ or $b$ are negative. Why is this so? That's a matter of calculus which maybe should be posted as its own question (which someone other than myself will have to answer.) But maybe the following will make it seem less mysterious.

You ask, why is $\int_0^1 \frac 1{\theta(1-\theta)}d\theta$ improper? By providing the indefinite integral, you supply most of the answer yourself:

\begin{align*} \int_0^1 \frac 1{\theta(1-\theta)}d\theta &= \left[log(\theta)-log(1-\theta)\right]_0^1\\ &= (log(1)-log(0)) - (log (0) - log(1))\\ &= (\infty) - (-\infty)\\ &= \infty \end{align*}

As for your question on notation, I haven't encountered that use of $\mapsto$ before either; but from context it seems clear that what they mean is something like "under the distribution". I.e., $``\theta\mapsto f_{\Theta|X}(\theta|0)"$ appears equivalent to "$\theta \sim f_{\Theta|X}(\theta|0)$", which can in turn be read as "$\theta$ distributed under the pdf $f_{\Theta|X}(\theta|0)$".