How to calculate the posterior in the Expectation-Maximization Algortihm from this example

Question

I'm trying to write a small program reproducing an example from this youtube video: Expectation Maximization: how it works $x_i$ is the points value

$b$ or any subsript $b$ is related to the blue distribution, and $a$ for the yellow distribution.

What I don't understand is how to calculate $b_i$ from this screenshot:

The video is a bit blurry so here's the equation: $$b_i=P(b|x_i)=\frac{P(x_i|b)P(b)}{P(x_i|b)P(b)+P(x_i|a)P(a)}$$

For $P(x_i|b)$ and $P(x_i|a)$ okay, that's just above, but what about $P(b)$ and $P(a)$?

Does it mean the probability that the given point has to be from one or the other distribution? Since there isn't only 2 in this example, it would be $0.5$ then? Regardless of any number of iterations through the EM algorithm ?

Or is it something else?

Thx in advance for any input,

Answer 1

Yes, you are right. $P(b)$ is the probability that overall any point comes from the distribution b, and likewise for $P(a)$.

It's hard to hear, but he muttered under his breath

• assume $P(b)=P(a)=0.5$ for now.

At the end, he explains you can also estimate the priors by using the initial $b_i$'s. So basically calculate all the $b_i$'s and then take their average, and of course $P(a)=1-P(b)$. You only do this once as this is the prior (do not update at every iteration).

Another option is to use your own beliefs/previous information:

• What proportion of points do you think should come from blue? Select this as $P(b)$.

In general, for simple algorithms like this the prior does not matter much into the Bayesian results. But if the data and model are complicated, a sensitivity analysis using different priors might be in order. (Here, though, you're likely to get the same results unless your priors are extremely extreme.)