A Markov chain $Q = \{ Q_0, Q_1, \ldots\}$ with state space and transition matrix as $U = \{1,2,3,4\}$ and
W = \begin{bmatrix} .2 & .3 & .3 & .2\\ .4 & .2 & .3 & .1\\ .1 & .1 & .7 & .1\\ .4 & .2 & .1 & .3 \end{bmatrix} . respectively. Find
$P\{Q_1 = 3, Q_2=2, Q_3 = 1, Q_4 =2, Q_5 = 2| Q_0 = 4\}$
What I did so far:
It's known that the Markov chain property is such that $P\{X_{n+1} = j| X_{0} = i_0, \ldots, X_{n+1}, X_n = 1 \} = P\{X_{n+1} = j | X_n =i \}$. So, I thought of using the property to solve the problem like this
$P\{Q_1 = 3| Q_0 = 4\} = p_{43} = 0.1$
Because the conditioned MC variable is $Q_0$, then, $P\{X_{n+1} = j | X_n =i \} = P\{Q_1 = 3| Q_0 = 4\}$ but I am not sure whether my approach approach is right. Similarly, $P\{ Q_3 = 3, Q_4 =1, Q_5 = 3| Q_2 = 1\} = P\{Q_3 = 3| Q_2 = 1\} = p_{13} = 0.3$
You're doing the right thing. $$P(Q_1 = 3, Q_2=2, Q_3 = 1, Q_4 =2, Q_5 = 2| Q_0 = 4)=p_{43}p_{32}p_{21}p_{12}p_{22}$$
However, your notation is incorrect. You don't meant to say $$P(Q_3 = 3, Q_4 =1, Q_5 = 3| Q_2 = 1) = P(Q_3 = 3| Q_2 = 1) = p_{13}$$ $p_{13}$ is just the first step in the calculation. The correct equation would be $$P(Q_3 = 3, Q_4 =1, Q_5 = 3| Q_2 = 1) =\\ P(Q_3 = 3| Q_2 = 1)P(Q_4=1,Q_5=3|Q_3=3)$$