Bayes' theorem and conditional probability?

Question

This is a bit of a soft-question, which I just happened to overhear, so please, bear with me.

"How can one derive Bayes’ theorem from the definition of conditional probability?"

After hearing said question, it got me thinking, and I tried to puzzle it out, but couldn't.

Could anyone kindly explain to me how this works?

I can't seem to wrap my head around it, so I would really appreciate it if someone could explain it to me as if I was about 10 years old mentally (which I'm starting to think I just might be).

Answer 1

Baye's Theorem is: $P(A\mid B) = \dfrac{P(B\mid A)\times P(A)}{P(B)}$

Conditional Probability is: $P(A \cap B) = P(A \mid B)\times P(B)$

---

Hint: begin with: $P(A\cap B)=P(B\cap A) \quad \because\text{ commutativity of }\cap$

Answer 2

When faced with this sort of question, first let us ask ourselves what you would like to derive. In this case, consider Baye's theorem: $$ P(A\mid B) = \dfrac{P(B\mid A)\cdot P(A)}{P(B)} $$

Next, consider what we know,or from where you want to get there. In our case, we would like to start with the conditional probability states that $ P( A \cap B)=P(A \mid B) \cdot P(B)$.

Now we can start doing what Mario Carneiro called "algebraic manipulations". We will start with the statement of conditional probability and end up with Baye's theorem.

$$ P( A \cap B)=P(A \mid B) \cdot P(B) \Leftrightarrow P(A \mid B)=\frac{P( A \cap B)}{P(B)} $$

Here, all we did was to divide both sides of the equation by $P(B)$. We notice that our equation now has the same form as the Baye's theorem: $P(A \mid B)= \frac {something}{P(B)} $.

With a little insight, we notice that the top term of the Baye's theorem, that is $P(B \mid A)P(A)$, is equal to $P(B \cap A)$.

This is where the commutativity the Graham evoked comes into play. It means that $P(A \cap B)=P(B \cap A)$. Therefor, we can replace the top term in our work in progress and we obtain: $P(A \mid B)=\frac{P(B\cap A)}{P(B)}$. We then expand the new top term and we end up with our (well, Baye's) theorem: since $P(B \cap A)=P(B\mid A) \cdot P(A)$, we replace $P(B\mid A)$ and we get... $$P(A \mid B)=\frac{P(B \mid A) \cdot P(A)}{P(B)}$$