A question on Random Walks and Markov Chain

184 Views Asked by At

Suppose that $ (X_n,n \in N) $ is a Markov chain whose transition probability matrix is given by

$ P=\begin{bmatrix}\alpha & 1-\alpha \\1-\beta & \beta \end{bmatrix} $

(a) Deduce that $ Z_n =(X_{n-1},X_n)$ is a Markov Chain with the four states $(0,0)$ $(0,1)$ $(1,0)$ $(1,1)$

(b) Determine the transition probability matrix of $ (Z_n,n \in N) $

1

There are 1 best solutions below

2
On

Ok, I'll assume that $(X_n)$ has state space $\{0,1\}$ and is time homogeneous.

What you have to prove in a) is the following: $P(Z_n=i_n|Z_{n-1}=i_{n-1},Z_{n-2}=i_{n-2}...Z_{0}=i_{0}) = P(Z_{n}=i_{n}|Z_{n-1}=i_{n-1})$, where $i_{k}$ is one of $\{(0,0), (0,1),(1,0), (1,1)\}$. This is called the Markov Property. It roughly means that the future depends only on the present, not the past.

a) This means that in your case, no matter what happened before the current $Z_n$, i.e. whatever the previous $Z_i$'s were, the transition probability depends only on $Z_{n}$. Try to formalize your argument for this particular case.

As for b) the transition probabilities $P(Z_n=i_n|Z_{n-1}=i_{n-1})$ mean what is the chance of $Z_{n}$ being equal to $i_{n}$ given $Z_{n-1}=i_{n-1}$ For example if $Z_{n-1}=(0,0)$ the transition probability $P(Z_n=(0,1)|Z_{n-1}=(0,0)) = P(X_{n-1}=0, X_{n}=1| X_{n-2}=0, X_{n-1}=0) = P(X_{n}=1|X_{n-1}=0)=1 -\alpha$

Note how we used the Markov property.

You will have to end up with a 4 by 4 matrix (1 row and 1 column for each state) and if you've done it correctly the rows will have to sum up to $1$. I can be of assistance if it's not too urgent.