Prove that $P(A|B) = P(A|B,C)P(C) + P(A|B,\overline C)P(\overline C)$

Question

Let $(Ω,S,P)$ be a probability space. Let $A, B, C ∈ S$ with $P(B)$ and $P(C) > 0$. If $B$ and $C$ are independent show that $$P(A|B) = P(A|B,C)P(C) + P(A|B,\overline C)P(\overline C)$$

$\textbf{My Solution:}$

[Note: All the arrows used in the solution represent the direction of flow of inference in a graphical model.]

[Note: I have used standard notation which are generally in the literature of probabilistic graphical models. Take a look. https://ermongroup.github.io/cs228-notes/representation/directed/]

It is given that A depends on B. So we can start with the fact that $B \rightarrow A$.

Take RHS $$P(A|B,C)P(C) + P(A|B,\overline C)P(\overline C)$$ $$ = \frac{P(A,B,C)}{P(B,C)}P(C) + \frac{P(A,B,\overline C)}{P(B,\overline C)}P(\overline C)$$ $$ = \frac{P(A,B,C)}{P(B)P(C)}P(C) + \frac{P(A,B,\overline C)}{P(B)P(\overline C)}P(\overline C)$$ $$ = \frac{P(A,B,C)}{P(B)} + \frac{P(A,B,\overline C)}{P(B)}\ \ \ \text{(eq. 1)}$$

$\textbf{Case I}: A \perp C$

from $(eq. 1)$ $$= \frac{P(A,B,C)}{P(B)} + \frac{P(A,B,\overline C)}{P(B)}$$ $$= \frac{P(A,B)P(C)}{P(B)} + \frac{P(A,B)P(\overline C)}{P(B)}$$ $$= \frac{P(A,B)}{P(B)}$$ $$= P(A|B) = LHS$$

$\textbf{Case II}: C\leftarrow A$

from $(eq. 1)$ $$= \frac{P(A,B,C)}{P(B)} + \frac{P(A,B,\overline C)}{P(B)}$$ $$= \frac{P(B)P(A|B)P(C|A)}{P(B)} + \frac{P(B)P(A|B)P(\overline C|A)}{P(B)}$$ $$= P(A|B)P(C|A) + P(A|B)P(\overline C |A)$$ $$= P(A|B) = LHS$$

$\textbf{Case III}: C\rightarrow A$

from $(eq. 1)$ $$= \frac{P(A,B,C)}{P(B)} + \frac{P(A,B,\overline C)}{P(B)}$$ $$= \frac{P(B)P(C)P(A|B,C)}{P(B)} + \frac{P(B)P(\overline C)P(A|B,\overline C)}{P(B)}$$ $$= P(A|B,C)P(C) + P(A|B,\overline C)P(\overline C)$$ How to proceed from here?

Answer 1

I send this question to my instructor, he answered. The answer is as follow:

$\textbf{Case III}:\ C \rightarrow A$

from (eq. 1)

$$={P(A,B,C) \over P(B)} + {P(A,B,\overline C) \over P(B)}$$ $$={P(A,B,C)P(C) \over P(B)P(C)} + {P(A,B,\overline C)P(\overline C) \over P(B)P(\overline C)}$$ $$={P(A,B,C) P(C) \over P(B,C)} + {P(A,B,\overline C) P(\overline C)\over P(B, \overline C)}$$ $$=P(A| B,C) P(C) + P(A|B,\overline C) P(\overline C)$$ $$=P(A| B) P(C) + P(A|B) P(\overline C)$$ $$=P(A| B) = LHS$$

Answer 2

I am kind of confused of your notations: what do you mean by $A\rightarrow C$ and $A\perp C$?

Actually from your (eq.1) there is only one line further to go: $$RHS=P(A,C\mid B)+P(A,\overline{C}\mid B)=P((A\cap C)\cup(A\cap\overline{C})\mid B)=P(A\mid B),$$

where the second equality follows from the fact that $(A\cap C)\cap(A\cap \overline{C})=\emptyset$.

Answer 3

Notice you ended up where you started in the end of part III. From the second line of part (III) $$\begin{split}\frac{P(B)P(C)P(A|BC)}{P(B)}+\frac{P(B)P(\bar C)P(A|B\bar C)}{P(B)}&=\frac{P(B|C)P(C)P(A|BC)}{P(B)}+\frac{P(B|\bar C)P(\bar C)P(A|B\bar C)}{P(B)}\\ &=\frac{P(C)P(A,B|C)}{P(B)}+\frac{P(\bar C)P(A,B|\bar C)}{P(B)}\\ &=\frac{P(A,B,C)}{P(B)}+\frac{P(A,B,\bar C)}{P(B)}\\ &=\frac{P(A,B)}{P(B)}\\&=P(A|B)\end{split}$$