Why $P(\theta, y \mid x) = P(\theta \mid x, y)P(y \mid x)$

68 Views Asked by At

Can any one give me any intuation like I'm 10 years old... why below is legal?

The definition of conditional probability states that the joint probability of two events $A$ and $B$ can be expressed as the product of the conditional probability of $A$ given $B$ and the probability of $B$: $$P(A, B) = P(A \mid B)P(B)$$

  • Thus $A=\theta$ and $B=y \mid x$, why $P(\theta \mid y \mid x) = P(\theta \mid x,y)$????

In step 2, we apply this definition to the joint probability $P(\theta, y \mid x)$ in two different ways. First, we can treat $\theta$ as event $A$ and $y$ as event $B$, giving us: $$P(\theta, y \mid x) = P(\theta \mid x, y)P(y \mid x)$$

Alternatively, we can treat $y$ as event $A$ and $\theta$ as event $B$, giving us: $$P(\theta, y \mid x) = P(y \mid x, \theta)P(\theta \mid x)$$

  • And how this one is come out?

Both of these expressions are valid ways to apply the definition of conditional probability to the joint probability $P(\theta, y \mid x)$.

1

There are 1 best solutions below

2
On BEST ANSWER

Arrange $A\cap B\cap C$ in two ways to see that:

$$(A\cap B)\cap C = A \cap (B\cap C)$$


Using the conditional probability definition you cite and treating $A\cap B$ as the first event and $C$ as the second event we see that

$$\Pr((A\cap B)\cap C) = \Pr(A\cap B\mid C)\Pr(C)$$

Now, do this again but this time with $A$ as the first event and $B\cap C$ as the second event but keep going and expand the resulting $\Pr(B\cap C)$ as well.

$$\Pr(A\cap (B\cap C)) = \Pr(A\mid B\cap C)\Pr(B\cap C) = \Pr(A\mid B\cap C)\Pr(B\mid C)\Pr(C)$$

Divide both expressions by $\Pr(C)$ and recall that they are equal. We have as a result:

$$\Pr(A\cap B\mid C) = \Pr(A\mid B\cap C)\Pr(B\mid C)$$

or, using the variable names you chose, replace $A$ with $\theta$, replace $B$ with $y$ and $C$ with $x$.