In the article here it had this question.
A walker moves on two positions a and b. She begins at a at time 0, and is at a next time as well. Subsequently, if she is at the same position for two consecutive time steps, she changes position with probability 0.8 and remains in the same position with probability 0.2; in all other cases she decides the next position by a flip of a fair coin. (a) Interpret this as a Markov chain on a suitable state space and write down the transition matrix P. (b) Determine the probability that the walker is in position a at time 10.
Let $X_n$ be the position that the walker is in nth time period. Then isn't the transition matrix is :
$$
\begin{matrix}
0.2 & 0.8 \\
0.1 & 0.5 \\
\end{matrix}
$$
The given answer says
The states are four ordered pairs aa, ab, ba, and bb
The given matrix is:
$$
\begin{matrix}
0.2 & 0.8 & 0 & 0\\
0 & 0 & 0.5 & 0.5 \\
0.5 & 0.5 & 0 & 0\\
0 & 0 & 0.8 & 0.2 \\
\end{matrix}
$$
How does it has these four states? Why is my answer wrong. Can someone please explain to me what I did wrong there.
The transition matrix must take into account not just the current position (that is to say, whether she is at $a$ or $b$ right now), but also the position she was at immediately before. This is because in order to know what the probability of switching positions is going to be, we must know both her current position and the position immediately preceding the current position. Therefore, we encode this as the set of all possible pairs of positions: $$(aa, ab, ba, bb).$$ Then, and only then, is the stochastic process a Markov chain, but we emphasize here that this is Markov on the space of pairs of positions, not individual positions, and each pair of positions is a unique state of the chain.
So, if we look at the pairs of positions, we know, for example, that if her last two positions is $aa$, she transitions to $ab$ with probability $0.8$. But she cannot transition from $aa$ to $ba$ nor $bb$, because the next pair of positions must begin with the same letter that the previous pair ended with. So the row corresponding to $aa$ should read $$\begin{bmatrix}0.2 & 0.8 & 0 & 0\end{bmatrix}$$ if we go by the ordering $(aa, ab, ba, bb)$ described above. The rest is straightforward.