What are "Filtering" and "Smoothing" with regards to Hidden Markov Models?

Question

The Wikipedia article about Hidden Markov Models mentions "filtering" and "smoothing" tasks, see here: http://en.wikipedia.org/wiki/Hidden_Markov_model#Filtering.

It gives a brief explanation but no motivating examples and no references to the equivalent algorithm or technique, if any exists.

For example, does filtering imply that for every hidden variable there is "error" state and we want to compute likelihood of our sensor being in that state in order to drop the measurement? Or is there something more to it?

In terms of missing links, e.g. I notice the http://en.wikipedia.org/wiki/Kalman_filter article that seems to mention HMM. Is "Kalman" proper way to refer to HMM based filtering mentioned above? Or are there lots of different types of filtering all based on HMM?

Answer 1

Filtering is when you are only allowed to use past data to make an estimate. Smoothing is when you are allowed to use both past and future data to make an estimate.

There are many filters for various types of HMM models. A Kalman Filter works on a linear-gaussian HMM model. A Markov process is one where all future states can be determined by only the current state, you do not need to know anything about the past history. A hidden-markov model is one where you don't observe the states directly but only some output dependent on the state.

If you want an example of a smoother which is the analog of the Kalman Filter, search for the Rauch-Tung-Striebel smoother. The smoother is actually a forward-backwards algorithm where you run the Kalman Filter forwards and then run the RTS smoother backwards to update the conditional distribution of the states given the future distribution of the state.

Answer 2

In the dynamic linear models literature, a distinction is often made between filtering, smoothing and predicting. At a given time $t$, filtering refers to the estimation of an unobserved state $x_t$ given an observed measurement $y_t$. Smoothing is the retroactive estimation of states $x_{t-k}, k \gt 0$ at past times, and predicting is the estimation of future states $x_{t+k}, k \gt 0$.