Calculating probability in Markov Chains

Question

I have a stationary and asymptotic markov chain $\ {X; t = 0,1,...}$ with state space $ Sx = { 1,2,3 } $ and transition matrix P \begin{bmatrix}0.2&0.5&0.3\\0.4&0.3&0.3\\1&0&0\end{bmatrix}with initial distribution $\overline p$ \begin{bmatrix}1/3\\1/3\\1/3\end{bmatrix}

I now want to compute $\ P(X_{3} = 3, X_{2} = 1|X_{1} = 2, X_{0} = 2)$.

The answer is $ P_{2,1} P_{1,3}$ = 0.12.

I have tried several approaches using conditional probability theorems but never seem to get the right answer.

I tried rearranging it to $ \frac{ P(X_{3}=3, X_{2}=1) P(X_{1}=2, X_{0}=2 | X_{3}=3, X_{2}=1)}{P(X_{1}=2, X_{0}=2)} $ and get $ P_{1,3} $

I would appreciate any help here please.

Answer 1

First of all, what you need to do is recall the Markov property. The Markov property tells you the following for any natural number $n$, and $k , x_1,...,x_n$ in the state space: $$ P(X_{n+1} = k | X_1 = x_1,...,X_{n-1} = x_{n-1}, X_n = x_n) = P(X_{n+1} = k | X_n = x_n) $$

That is, only "the present" determines the future : the future is independent of the past at each point.

To make use of this, since you have multiple $X_i$ on the left side of the $|$, you must break it up using the fact $$P(A \cap B | C) = \frac{P(A \cap B \cap C)}{P(C)} = \frac{P(A \cap B \cap C)}{P(B \cap C)}\frac{P(B \cap C)}{P(C)} = P(A | B,C)P(B|C)$$

(Note that $B,C$ is the same as $B \cap C$, it is just that it is notationally easier on the right side of the $|$ to write the comma)

So we start with $K = P(X_3 = 3 ,X_2= 1| X_1 = 2, X_0 = 2)$. Let us do this break up for $A \to X_3 = 3$ and $B \to X_2 = 1$ and $C \to X_1 = 2 , X_0 = 2$ : $$ K = P(A\cap B| C) = P(A | B,C)P(B|C) \\ = \color{green}{P(X_3 = 3 | X_2 = 1 , X_1 = 2 , X_0 = 2)}\color{blue}{P(X_2 = 1 | X_1 = 2 , X_0 = 2)} $$

Now, use the Markov property on the green expression.You get $\color{green}{P(X_3 = 3 | X_2 = 1)} = 0.3$ since this is the transition probability from state $1$ to state $3$.

Similarly ,the blue expression is $\color{blue}{P(X_2 = 1 | X_1 = 2)} = 0.4$ since this is the transition probability from state $2$ to state $1$.

Their product is $0.12$, as desired.

Answer 2

As an aid, you may find it useful to draw out the state diagram. For instance, you might obtain something like the following:

From this diagram, you can see that if you're in state $X_1 = 2$ (at $t = 1$), then the probability of $X_2 = 1$ (at $t = 2$) is $P_{2,1} = 0.4$, and the probability of going from there to $X_3 = 3$ (at $t = 3$) is $P_{1,3} = 0.3$. The desired joint probability is just the product of the two individual probabilities, because the transitions are independent:

$$ P(X_3 = 3, X_2 = 1 \mid X_1 = 2, X_0 = 2) = 0.4 \cdot 0.3 = 0.12 $$