Calculate $f_i$ for all in state spaces of the Markov chain depicted by the transition matrix.

73 Views Asked by At

I just start learning about Markov processes. Specify the classes of the following Markov chain, and determine whether they are transient or recurrent, then calculate $f_i$ for all in state spaces of the Markov chain depicted by the transition matrix. Here, for any state $i$ we let $f_i$ denote the probability that, starting in state $i$, the process will ever reenter state $i$.

$\textbf{P} = \begin{Vmatrix} \frac{1}{4} & \frac{3}{4} & 0 & 0 & 0\\ \\ \frac{1}{2} & \frac{1}{2} & 0 & 0 & 0\\ \\ 0 & 0 & 1 & 0 & 0\\ \\ 0 & 0 & \frac{1}{3} & \frac{2}{3} & 0\\ \\ 1 & 0 & 0 & 0 & 0\\ \end{Vmatrix},$

This is what I get for the classes and if they were recurrent or transient:

$S_1$ = {0,1}, recurrent. $S_2$ = {2}, recurrent. $S_3$ = {3}, transient. $S_4$ = {4}, transient.

However, I am having problems calculating $f_{i}$. Can someone help me?