I saw this sketchy equation and wondered what is correct or wrong and which assumptions are needed for (1) and (2) (maybe it is due to unfamiliar/sloppy notation):
$$\mathbb{E}[A|B=b] \overset{(1)}{=} \int a P(a|B=b) = \int a P(B=b|a) P(a) \overset{(2)}{=} \mathbb{E}[a P(B=b|a)]$$
(1) I thought I would need the conditional density $f_A(\centerdot|B=b)$, but looks intuitively ok. What is required for this to hold (if in any case it holds)?
(2) I do not get this. I would expect Fubini, but..
Step-by-step filling in details:
$$\mathbb{E}[A|B=b] \overset{(1)}{=} \int_\Omega a P(A=a|B=b) \propto \int_\Omega a P(B=b|A=a) P(A=a) \overset{(2)}{=} \mathbb{E}[a P(B=b|A=a)]$$
- $A$ and $B$ are random variables
- I assume that the integral are proportional, since according to Bayes the divisor is missing.
- $a$ inside $P$ meant the event $A=a$
- I added $\Omega$ as sample space to the integral, then the Integral should be $dP$? I am not sure.. Maybe the integral is in the pushforward measure of $A$.
Assume that $A$ and $B$ have densities $f_A$ and $f_B$ respectively and that $A$ conditionally on $B$ and $B$ conditionally on $A$ have conditional densities $f_{A\mid B}(\ \mid\ )$ and $f_{B\mid A}(\ \mid\ )$ respectively. Then the joint density $f_{A,B}$ of $(A,B)$ exists and is such that, for almost every $(a,b)$, $$ f_{A,B}(a,b)=f_{A\mid B}(a\mid b)f_B(b)=f_{B\mid A}(b\mid a)f_A(a). $$ Thus, for every measurable function $u$, $$ E[u(A)\mid B=b]=\int_\mathbb Ru(a)f_{A\mid B}(a\mid b)\mathrm da=f_B(b)^{-1}\int_\mathbb Ru(a)f_{B\mid A}(b\mid a)f_A(a)\mathrm da, $$ where the first = sign is a definition and the second = sign follows from the identity above. Consequently, the "Step-by-step filling in details" suggested in the question is mostly correct, with three exceptions. First, the integrals are not on $\Omega$ but on the image set of $A$, which we assumed to be $\mathbb R$. Second, when the random variables $A$ and $B$ have densities, every $P(A=a|B=b)$, $P(B=b|A=a)$ and $P(A=a)$ is actually zero. Third, notations such as $E[aP(B=b|A=a)]$ cannot be recommended.