I just read a very nicely written introduction paper for the expectation maximisation algorithm published in Nature biotechnology by Do and Batzoglou (http://www.nature.com/nbt/journal/v26/n8/full/nbt1406.html).
Regarding the example shown in the paper, which is a coin toss problem, I have simple question, what if I start with $\hat{\theta}_{A}^{(0)}=0.5$ and $\hat{\theta}_{B}^{(0)}=0.5$? Looks like the algorithm doesn't work since the next parameter will remain same.
Am I wrong with such an initial value issue?
or Is this one of the critical issue in the expectation maximisation algorithm?
You're right. There's an inherent symmetry in the problem that needs to be spontaneously broken, and if you start with a symmetric guess the symmetric update procedure necessarily reproduces the symmetric guess. This not a problem, though, as long as the procedure converges to the same asymmetric solution as long as you choose asymmetric initial values.