I have a Markov chain $\{X_n\}$, with the states $S=\{0,1\}$ and the probability matrix
$$p_{11}=1-a \quad p_{12}=a \quad p_{21}=b \quad p_{22}=1-b \quad 0<a, b<1$$
I want to show that the stochastic process $Z_n=(X_{n-1},X_n) \quad n \geq 1,$ is the Markov chain and write its probability matrix.
I don't know where to start and what does it really mean that $Z_n=(X_{n-1},X_n)$. Thank you for your help!
There are $4$ states for $Z_n$, namely $$ (1,1),\;\;(1,2),\;\;(2,1),\;\;(2,2) $$ For example, $Z_n$ is in state $(1,2)$ when $X_{n-1}=1$ and $X_n=2$.
Note that the value of $X_{n-1}$ plays no role in the transition from $Z_n=(X_{n-1},X_n)$ to $Z_{n+1}=(X_n,X_{n+1})$.
Based on that observation, I've filled in the second row of the transition probability matrix for $Z_n$. \begin{array}{|c|c|c|c|c|} \hline &(1,1)&(1,2)&(2,1)&(2,2)\\ \hline (1,1)&&&&\\ \hline (1,2)&0&0&b&1-b\\ \hline (2,1)&&&&\\ \hline (2,2)&&&&\\ \hline \end{array} In the above example, from state $(1,2)$, the next state must be either $(2,1)$ or $(2,2)$, and the probabilities for those two outcomes are $p_{21}$ and $p_{22}$ respectively.
See if you can fill in the rest.