Markov chains for beginners, how to think about them?

Question

So this is what my book states:

Random variables $X,Y, and Z$ are said to form a Markov chain in that order denoted $X\rightarrow Y \rightarrow Z$ if and only if:

$p(x,y,z)=p(x)p(y|x)p(z|y) $

That's great and all but that doesn't give any intuition as to what a Markov chain is or what it implies.

Can someone please give me more intuition as to how I should think about Markov chains?

Thanks a lot!!

Answer 1

Markov chains have the Markov Property, quoting Wikipedia:

A stochastic process has the Markov property if the conditional probability distribution of future states of the process depends only upon the present state, not on the sequence of events that preceded it.

Therefore state $Y$ depends only on $X$, hence $p(y) = p(y|x)p(x)$.

And state $Z$ depends only on $Y$, hence $p(z) = p(z|y)p(y)$.

In the case of Markov chains, the conditional probability $p(x_{t+1}|x_t)$ is referred to as the transition probability, and can be intuitively thought of as: given I'm in state $X_{t}$ what is the probability I end up in state $X_{t+1}$. It is therefore a random process with a limited memory of only one step behind it.

Answer 2

The definition you recalled means that $Z$ depends on $(X,Y)$ only through $Y$, that is, $p(z\mid x,y)=p(z\mid y)$.

A typical example of a Markov chain is given by the positions $X$, $Y$ and $Z$ of a random walk on the integer line $\mathbb Z$ after $1$, $2$ and $3$ steps. If you are given $(X,Y)$ and want to make predictions about $Z$, only $Y$ matters.

Edit: A reformulation of the Markov property is that $p(x,z\mid y)=p(x\mid y)p(z\mid y)$, that is, that $x$ and $z$ are independent conditionally on $y$. This last formulation shows the symmetry of the Markov property with respect to the arrow of time.