Find the variance of Y .

Question

I have conflicting answers with different methods. Here is the question:

The number of defects per yard in a certain fabric, Y , was known to have a Poisson distribution with parameter $\lambda$. The parameter $\lambda$ was assumed to be a random variable with a density function given by:

$f (\lambda) = e^{−\lambda}, \text{for } \lambda≥0,$

Find the expected number of defects per yard by first finding the conditional expectation of Y for given $\lambda$.

Since $\lambda$ is the expected value of any random variable with a poisson distribution, it follows that $E(Y | \Lambda = \lambda) = \lambda$. Since $Y$ has a poisson distribution, it follows that $E(Y) = E(\lambda) = \lambda$

We have a theorem which states that $E(E(Y | \Lambda = \lambda)) = E(Y).$

$$E(Y) = E(\Lambda) = \int^\infty_0E(Y | \Lambda = \lambda) e^{-\lambda}dλ$$ $$E(Y) = E(\Lambda) = \int^\infty_0\lambda e^{-\lambda}dλ$$ $$E(Y) = E(\Lambda) = 1$$

Thus, $E(Y) = 1$

Find the variance of Y.

It follows that the variance of any random variable with a poisson distribution is $\lambda$.

Thus,$V(Y) = 1$

Using the theorem below, I get the following answer instead $V(Y) = 2$

$V(Y_1)=E[V(Y_1|Y_2)] +V[E(Y_1|Y_2)].$

Applying it to our case, we have

$V(Y)=E[V(Y_1|\Lambda)] +V[E(Y_1|\Lambda)].$

$V(Y)=E[\Lambda] +V[\Lambda].$

$V(Y) = 2$

I am leaning more towards the second answer

Answer 1

1) You stated the theorem, but you didn't use it. It goes like this: $$E[Y] = E\{E[Y|\lambda]\} = E\{\lambda\} = 1.$$

2) \begin{align*} Var[Y] &= Var\{E[Y|\lambda]\}+E\{Var[Y|\lambda]\}\\ &=Var\{\lambda\}+E\{\lambda\}\\ &=1+1\\ &=2 \end{align*} I forgot the name of this property, but this is the one you should use. $Y$ by itself does not necessarily follow a Poisson distribution with rate $\lambda$. However $Y|\lambda$ does, since it was given.

Answer 2

$Y$ does not have a poison distribution with parameter $1$ so the first answer does not apply.

$Y$ has a conditional poison distribution when given parameter $\lambda$, which itself has an exponential distribution.   So the second method gives the answer. (PS: It's the Law of Iterated Variance.)

---

For verification, look at the marginal probability mass function of $Y$.

$\begin{align}\mathsf P(Y{=}y) & = \int_0^\infty f_\lambda(h)\; \mathsf P(Y{=}y\mid \lambda{=}h)\operatorname d h\;\mathbf 1_{y\in\Bbb N} \\[1ex] & = \int_0^\infty \frac{e^{-2h}h^y}{y!}\operatorname d h\;\mathbf 1_{y\in\Bbb N} \\[1ex] & = \big(\tfrac 12\big)^{y+1}\;\mathbf 1_{y\in\Bbb N} \end{align}$

$\therefore Y$ has a 0-indexed geometric distribution with success parameter $\tfrac 12$. $$Y\sim\mathcal{Geo}_0(\tfrac 1 2)\\ \mathsf E(Y) = \tfrac {1-\tfrac 12}{\tfrac 1 2}=1\\ \mathsf{Var}(Y)=\tfrac {1-\tfrac 12}{\big(\tfrac 1 2\big)^2}=2$$

Answer 3

Yes - I too would go with your second answer, for the reason you give.

$Var(Y|\lambda=1)=1$ but $\lambda$ is uncertain so $Var(Y)$ should be bigger than this.

As an empirical test, try this R code:

> cases <- 10000000
> set.seed(2015)
> lambda <- rexp(cases, rate=1)
> y <- rpois(cases, lambda)
> mean(lambda)
[1] 0.9999552
> sd(lambda)^2
[1] 0.9999436
> mean(y)
[1] 1.000106
> sd(y)^2
[1] 2.000172