Relationship between conditional expectation and marginal join?

Question

I have seen in book (Statistical Rethinking) this equation: $$ Pr(w) = E(Pr(w|p)) = \int Pr(w|p)Pr(p)dp $$ Where $$ Pr(w, p) $$ is join probably density function. Can somebody explain me the equality between expected value of this conditional probability and marginal density?

Answer 1

Let $W,P$ be random variables with joint PDF $f_{W,P}\left(w,p\right)$.

The marginal PDF's can be found as: $$f_{W}\left(w\right)=\int f_{W,P}\left(w,p\right)dp\text{ | and | }f_{P}\left(p\right)=\int f_{W,P}\left(w,p\right)dw$$

For a fixed $p$ and under suitable conditions we can define a conditional PDF: $$f_{W}\left(w\mid p\right)=\frac{f_{W,P}\left(w,p\right)}{f_{P}\left(p\right)}$$

So we have the equality: $$f_{W,P}\left(w,p\right)=f_{W}\left(w\mid p\right)f_{P}\left(p\right)$$ Taking integrals on both side we find: $$f_{W}\left(w\right)=\int f_{W,P}\left(w,p\right)dp=\int f_{W}\left(w\mid p\right)f_{P}\left(p\right)dp$$

Here $\int f_{W}\left(w\mid p\right)f_{P}\left(p\right)dp$ can be recognized as $\mathbb{E}f_{W}\left(w\mid P\right)$ justifying that:$$f_{W}\left(w\right)=\mathbb{E}f_{W}\left(w\mid P\right)=\int f_{W}\left(w\mid p\right)f_{P}\left(p\right)dp$$

So in the notation used in your question:$$\Pr(w)=\mathbb E\Pr(w\mid P)=\int\Pr(w\mid p)\Pr(p)dp$$

The only difference is in the fact that I maintain a capital $P$ in the notation of the expectation. This to avoid confusion between random variable $P$ and the values $p$ that this variable can take. Quite often neglecting this difference leads to confusion.