Computing the transition matrix of a Markov chain yielded from another Markov chain.

62 Views Asked by At

Let {$Xt;t = 0, 1, . . .$} be a Markov chain with state space $S_X=${$0, 1, 2$}, initial distribution $p$, and transition matrix $P$, where $P$ is

\begin{bmatrix}0&2/3&1/3\\1/4&0&3/4\\1/2&1/2&0\end{bmatrix}

Assume that the initial distribution of the chain is $\pi=\frac{1}{11}(3,4,4)$.

Let $Y_t=X_{10−t}$ for $t = 0, 1, . . . , 10.$ Then, {$Y_t;t = 0, 1, . . . , 10$} is also a Markov chain.

I am tasked to compute the transition matrix of {$Y_t;t = 0, 1, . . . , 10$}.

I understand that this is done using

$P(Y_1=j|Y_0=i)=\frac{P(Y_0=i|Y_1=j)P(Y_1=j)}{P(Y_0=i)}=\frac{P(X_{10}=i|X_9=j)P(X_9=j)}{P(X_{10}=i)}=\frac{P_{j,i}\pi_j}{\pi_i}$

which should yield the transition matrix

\begin{bmatrix}0&1/3&2/3\\1/2&0&1/2\\1/4&3/4&0\end{bmatrix}

However, I don't understand how to interpret $\frac{P_{j,i}\pi_j}{\pi_i}$ and how to translate this into the transition matrix. Does anyone know how to help? How do I find $\pi_j$?

1

There are 1 best solutions below

8
On

First note that the initial distribution $\pi$ is the unique stationary distribution for $P$, that is, $\pi=\pi P$. In general, a Markov chain with stationary distribution $\pi$ is reversible if and only if $\pi_i P_{i,j} = \pi_j P_{j,i}$ for all states $i$,$j$ - which we can verify here by a straightforward computation.

The transition probabilities $\hat P_{i,j}=\frac{\pi_j}{\pi_i}P_{j,i}$ are that of the reversed chain.