Stochastic matrix first return time question

38 Views Asked by At

Calculate the first return probability for the following states in the matrix.

enter image description here

Where the states are 1,2 and 3.

for $f_{33}^{1}=p_33=0$

for $f_{33}^{2}=p_{32} p_{23}=(1)*(1/2)$

for $f_{33}^{3}=p_{32}p_{23}p_{33}=0$

for $f_{33}^{4}=p_{32}p_{23}p_{32}p_{23}=1*(1/2)(1)(1/2)=1/4$

So I get $f_{33}^{2n}=(1/2)^n$

and $f_{33}^{2n-1}=0$

But I am not sure how to find $f_{11} $ and $f_{22}$

1

There are 1 best solutions below

1
On BEST ANSWER

Assuming that the matrix that you presented is $P^T$, then let us start with $f_{22}$. Once the chain is in state $2$ it either goes (w.p. $1/2$) to $1$ or $3$ and then w.p. $1$ goes back to $2$, hence, $$ f_{22}(2) = \frac{1}{2}\times 1 + \frac{1}{2}\times 1 = 1, $$ and $0$ for any other $n$. As for $f_{11}$ - note that this is a periodic chain with period $2$, hence the chain can return to $1$ with any number of $2n$ steps, in particular $$ f_{22}(2n) = \frac{1}{2^n}\,. $$ I.e., the chain goes to state number $2$ in the first step and then alternates $n\ge 0$ times between $2$ and $3$ until it goes back from $2$ to $1$.