$\mathbb{P} (X_3=3|X_0=1)$ in a Markov chain

72 Views Asked by At

Given a Markov chain with state $[1,2,3,4]$:

$$ \begin{matrix} \frac{1}{5} & \frac{2}{5}& \frac{1}{5} & \frac{1}{5}\\ \frac{3}{4} & \frac{1}{4} & 0 & 0 \\ \frac{1}{8} & \frac{3}{8} & \frac{3}{8} &\frac{1}{8} \\ 0 & \frac{1}{3} & 0 & \frac{2}{3}\\ \end{matrix} $$

I need to find $\mathbb{P}(X_3=3|X_0=1)$.

I tried

$$ \begin{matrix} 0.008 & 0.421875 & 0.001953125& 0.00000000\\ 0.064 & 0.015625 & 0.052734375 & 0.03703704\\ 0.008 & 0.000000 & 0.052734375 & 0.00000000\\ 0.008 & 0.000000 & 0.001953125 & 0.03703704\\ \end{matrix} $$

I don't want to compute this manually. I thought of a simpler way:

$1 \rightarrow 1 \rightarrow 1 \rightarrow 3$

$1 \rightarrow 2 \rightarrow 1 \rightarrow 3$

Even then I don't know how to proceed. The answer is $\frac{17}{250}$. Even after cubing my matrix, don't get it. And I would like to know the closed irreducible set and transient set if any?

1

There are 1 best solutions below

2
On

The above comment is correct, I'm just going to offer some additional explanation. To be honest I don't fully understand what you tried to do so far.

Indeed, you could try to calculate this by hand, but then you need to account for all possible paths that lead from $1$ to $3$ in 3 steps, and this is going to be tedious.

Instead, notice that if you pre-multiply your matrix $P$ by: $e_1 = \begin{bmatrix}1 & 0& 0& 0 \end{bmatrix}$ you will have the vector of probabilities of each state at $t$, given that at $t-1$ you were at $1$. If you then post-multiply it by you $P$ again, you will obtain vetor of probabilities of each state at $t=2$, and so on. Therefore, the probability of reaching state $3$ after $3$ steps, given that you started at $1$ will be: $$\mathbb{P}(X_3=3|X_0=1)=e_1\times P \times P \times P \times e^T_3 = e_1 \times P^3 \times e^T_3 = P^3_{13}$$