Markov Transient State using Stationary Distribution

144 Views Asked by At

I have the following Discrete Time Markov Chain with $a^{(0)} = \ $$ \begin{pmatrix} 0 & 0 & 0 & 1 & 0 \end{pmatrix} $
$$ P = \begin{pmatrix} 0 & 1/2 & 1/2 & 0 & 0 \\ 1/2 & 0 & 0 & 1/2 & 0 \\ 1/2 & 0 & 0 & 1/2 & 0 \\ 0 & 1/4 & 1/4 & 0 & 1/2 \\ 0 & 0 & 0 & 1 & 0 \\ \end{pmatrix} $$

Using the fact that this DTMC is irreducible and periodic with $k=2$ , I found the stationarydistribution as: $$ \begin{pmatrix} 1/6 & 1/6 & 1/6 & 1/6 &2/6 \end{pmatrix} $$

I am now asked to figure out $ a^{(3)} $ , which I know is defined as $ a^{(0)}P^3 $, but I am asked specifically to do this without calculating $ P^3 $. Can I use the stationary distribution to show that $ a^{(3)} $, in this case, equals $$ \begin{pmatrix} 0 & 5/16 & 5/16 & 0 &3/8 \end{pmatrix} \ \ ?$$

1

There are 1 best solutions below

0
On

We do not need to compute $P^3$ explicitily.

$$a^{(0)}P^3=(((a^{(0)})P)P)P$$

The computation involves $3$ vector-matrix product.

That is first we compute $a^{(1)}$ then use it to compute $a^{(2)}$ and finally use it compute $a^{(3)}$.