Integrate the likelihood function

Question

We know that for a density function $f(x\mid\theta)$ we have $$\int^{\infty}_{-\infty}f(x\mid\theta)\, dx=1$$ Do we also have for the likelihood function $L(\theta\mid x)$ that $$\int^{\infty}_{-\infty}L(\theta\mid x)\, d\theta=1?$$ My guess is yes, since $L(\theta\mid x)$ is some kind of "possibility" of the parameter that under different data.

Thanks~

Answer 1

Take the Poisson distribution example, with two data, say $x_1=1,\,x_2=2$.

We have $$f(x\mid \mu)=\frac{\mu^{x}e^{-\mu}}{x!}$$ so $$L(\mu\mid x_1, x_2)=\prod_{i=1}^2\frac{\mu^{x_i}e^{-\mu}}{x_i!}=\frac{\mu^3e^{-2\mu}}{2}$$ Then $$\int_{-\infty}^\infty L(\mu\mid x_1,x_2)\,d\mu=\frac12\int_0^\infty\mu^3e^{-2\mu}\,d\mu=\frac12\cdot\frac38\neq1.$$

Answer 2

A biased coins turns up "heads" $2/3$ of the time; a fair coin $1/2.$

Given a "head", the likelihood function is \begin{align} & L(\text{fair}) = 1/2, \\ & L(\text{biased}) = 2/3. \end{align} Do these add up to $1$? Not if we assign weight $1$ to each possible coin.

And assigning such weights does not appear to make sense except when they are prior probabilities. Suppose we have prior probabilities $\Pr(\text{fair}) = 0.9 = 1 - \Pr(\text{biased}) = 1 - 0.1\,.$ Then integrating our likelihood yields $$ (0.9)(2/3) + (0.1)(1/2) = 0.65 $$ and that is not $1,$ but rather it is the marginal probability of the observed data, i.e. the marginal probability of "heads".

Answer 3

The meaningful way to turn a likelihood into a probability is to integrate it against a prior probability distribution for $\theta$. Good choices of a prior are more art than science. But in particular, the Lebesgue measure on $\mathbb{R}$ is not admissible for this purpose, because it's not a probability distribution. However, if you have $\theta$ confined to $[a,b]$, then you can use the uniform distribution on $[a,b]$ as a "naive prior". You could do the same if $\theta$ is confined to a finite set.

When you do this, you still don't get $1$, though, you get the probability of observing your particular data set under your prior distribution for $\theta$. For example, consider a Bernoulli($p$) data point $x_1=1$ and the naive prior for $p$. The likelihood function is $L(p \mid \{ 1 \})=p$, and the integral of this against the naive prior is $\int_0^1 p \, dp = 1/2$.