Discrete-time Markov Chain; $n$-step transitions

54 Views Asked by At

Let $\{X_{n}\}_{n\geq0}$ be a discrete-time Markow chain on the state space $S=\{1,2,3\}$ with transition matrix

\begin{pmatrix} 1/3 & 1/3 & 1/3 \\ 0 & 2/3 & 1/3 \\ 2/3 & 1/3 & 0 \end{pmatrix}

The initial distribution is $\mathbb{P}(X_{0}=1)=\mathbb{P}(X_{0}=2)=\mathbb{P}(X_{0}=3)=1/3$. What is $\mathbb{P}(X_{3} \in \{2,3\})$?

My answer:

First of all we note that $\mathbb{P}(X_{3} \in \{2,3\})=\mathbb{P}(X_{3} = 3)+\mathbb{P}(X_{3} = 2)$. We know that for $n$-step transition probabilities the unconditional form (given the initial distibution) has:

$\mathbb{P}(X_{n}=j)=\displaystyle\sum_{k \in S} \lambda_{k}(P^{n})_{kj}$,

where $\mathbb{P}(X_{0}=i)=\lambda_{i}$.

Using the above we get:

$\mathbb{P}(X_{3}=2)=\displaystyle\sum_{k=1}^{3}\lambda_{k}(P^{3})_{k2}=\lambda_{1}P_{12}^{3}+\lambda_{2}P_{22}^{3}+\lambda_{3}P_{32}^{3}=1/3(13/27+14/27+13/27)=40/81$

Similarly:

$\mathbb{P}(X_{3}=3)=\displaystyle\sum_{k=1}^{3}\lambda_{k}(P^{3})_{k3}=\lambda_{1}P_{13}^{3}+\lambda_{2}P_{23}^{3}+\lambda_{3}P_{33}^{3}=1/3(7/27+7/27+6/27)=20/81$.

Thus $\mathbb{P}(X_{3} \in \{2,3\})=40/81+20/81=60/81$

For completeness $\mathbb{P}(X_{3}=1)=21/81$, as expected. Is that correct?

2

There are 2 best solutions below

1
On BEST ANSWER

I haven't checked your workings, as firstly I don't feel that's really the point of the site, but maybe more importantly that's not the point of Markov chains in general.

My suggestion is to not the following: $$ \{ X_3 \in \{2,3\} \} = \{ X_3 = 1 \}^c. $$ Now, it should be easier, computationally, to find this, since there is only one option. In particular, you only need to find the first column of $P^3$.

To answer a question you may be thinking of... no, I can't see an easier way than just calculating directly. Sometimes there is, but other than replacing $\{2,3\} = \{1\}^c$, I can't see anything.

1
On

Looks right to me. I did a quick check in Mathematica: With[{a={{1,1,1},{0,2,1},{2,1,0}}/3},{1,1,1}.a.a.a/3], which gives the result {7/27,40/81,20/81}.