Is the EM-algorithm the same thing that variational inference in LDA?

398 Views Asked by At

I am new in the probabilistic topic modeling, and I need to understand deeply the LDA process, I understand what want to do the inference process in LDA, and I understand too that there is 2 "types" of inference: probabilistic methods (like the Gibbs samppling) and deterministic ones, but in the deterministic type some papers talk about Expectation Maximization algorith and some others talk about something called "variational inference". So, are those thing the same?

Also, can you explain how is applied the EM algorithm for the LDA?

1

There are 1 best solutions below

0
On BEST ANSWER

Variational inference in this context refers to Bayesian variational inference, which is indeed a form of EM with an additional assumption that a certain posterior distribution that appears in LDA is factorable. You'll find an overview of this and all other known ways of calculating LDA in this paper: On Smoothing and Inference for Topic Models - Asuncion et al.. The conclusions of the paper are that most of these methods for LDA are essentially the same if you tune your hyper parameters correctly, with collapsed variational Bayes being the preferred choice in terms of speed and efficiency.