I’m trying solve this Problem: Let $(X_n)_{n\geq 0}$ a Markov chain with States set $S=\{0,1\}$ and transition probability matrix $$P=\left( \begin{array}{cc} 1-a& a\\ b& 1-b \end{array} \right).$$ Define a new stochastic process $(Z_n)_{n\geq 1}$ by $Z_n=(X_{n-1},X_n)$. Argue that $(Z_n)_{n\geq 1}$ is a Markov chain and write down its transition matrix.
My solution
First I found the states space of $(Z_n)_{n\geq 1}$, which clearly is $E=\{(0,0),(0,1),(1,0),(1,1)\}$.
For prove that is Markov chain I verified that satisfy the Markov property as follow $$ \begin{split} P(Z_{n+1}=(i_n,i_{n+1})|Z_{1}&=(i_0,i_{1}),Z_{2}=(i_1,i_{2}),\ldots,Z_{1}=(i_{n-1},i_{n}))=\\ &=\dfrac{P((X_n,X_{n+1})=(i_n,i_{n+1}),(X_0,X_1)=(i_0,i_{1}),\ldots, (X_{n-1},X_n)=(i_{n-1},i_n))}{P((X_0,X_1)=(i_0,i_{1}),\ldots, (X_{n-1},X_n)=(i_{n-1},i_n))}\\ &=\dfrac{P(X_{n+1}=i_{n+1},X_n=i_n,\ldots,X_0=i_0)}{P(X_n=i_n,\ldots,X_0=i_0)}\\ &=\dfrac{P(X_{n+1}=i_{n+1}|X_n=i_n,\ldots,X_0=i_0)P(X_n=i_n,\ldots,X_0=i_0)} {P(X_n=i_n,\ldots,X_0=i_0)}\\ &=P(X_{n+1}=i_{n+1}|X_{n-1}=i_{n-1},X_n=i_n)\\ &=P(X_{n+1}=i_{n+1},X_n=i_n|X_{n-1}=i_{n-1},X_n=i_n)\\&=P(Z_{n+1}=(i_n,i_{n+1})|Z_n=(i_{n-1},i_n)), \end{split} $$ I supposed that the states have to be consecutive because in other case the $P(X_n=i_n,\ldots,X_0=i_0)=0$. Therefore $(Z_n)$ is a Markov chain. Now my problem for find the transition probabilities:
My problem
For find transition probabilities I have problems because the rown en transition matrix for $(Z_n)$ doesn't sum 1 and seems the transitions probabilies are the same of the chain $(X_n)$. For instante, consider
$P(Z_2=(0,0)|Z_1=(0,0))=P(X_2=0|X_1=0,X_0=0) = P(X_2=0|X_1=0)=1-a$ (Using Markov property)
$P(Z_2=(0,0)|Z_1=(1,0))=P(X_2=0|X_1=0,X_0=1) = P(X_2=0|X_1=0)=1-a$
$P(Z_2=(0,0)|Z_1=(1,1))=P(X_2=0|X_1=1,X_0=1) = 0$
$P(Z_2=(0,0)|Z_1=(0,1))=P(X_2=0|X_1=1,X_0=0) = 0$
Someone can explain me what's wrong with my solution and help me to solve that please.