Let $(X_n)_{n \in N}$ be a Markov chain with the set of states $I=\{ 0,1,2,3\}$ and with the given transition matrix:
$$P=\begin{bmatrix} \frac{1}{4} & \frac{3}{4} & 0 & 0 \\ \frac{1}{2} & \frac{1}{4} & 0 & \frac{1}{4} \\ \frac{1}{3} & 0 & \frac{1}{3} & \frac{1}{3} \\ 0 & \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \end{bmatrix} $$
The probability $P\{ X_5=1, X_3=1, X_2=3| X_1 = 2\}$ is being questioned.
I have not done this type of question. There are a couple like this. I believe this might not be too dificult to understand, but I just do not know how to go about solving this.
First applying Bayes theorem and then the Markovian property: $$P[X_5=1,X_3=1,X_2=3|X_1=2] = P[X_5=1,X_3=1|X_2=3,X_1=2] \cdot P[X_2=3|X_1=2] = P[X_5=1,X_3=1|X_2=3] \cdot P[X_2=3|X_1=2] $$ Repeating this then gives $$P[X_5=1,X_3=1,X_2=3|X_1=2] = P[X_5=1|X_3=1]\cdot P[X_3=1|X_2=3]\cdot P[X_2=3|X_1=2]$$
In fact the transition matrix doesn’t change, so we can rewrite this as $$P[X_5=1,X_3=1,X_2=3|X_1=2] = P[X_{n+2}=1|X_n=1]\cdot P[X_{n+1}=1|X_n=3]\cdot P[X_{n+1}=3|X_n=2]$$
The $i,j$th element of the state transition matrix represents the probability of a transition to state $j$ from state $i$. Therefore we can write $$\left(P\right)_{i,j} = P(X_{n+1}=j|X_n=i)$$ In fact, the $i,j$th element of the $k$th power of the state matrix gives the probability of a transition from $i$ to $j$ at the $k$th transition. $$\left(P^k\right)_{i,j} = P(X_{n+k}=j|X_n=i)$$ Hence you should compute the transition matrix squared and then read off the coefficients: $$P[X_5=1,X_3=1,X_2=3|X_1=2] = \left(P^2\right)_{1,1} \left(P\right)_{3,1} \left(P\right)_{2,3} $$