Calculate $E[N_{i}]$ and then $f_i$ for all in state spaces of the Markov chain depicted by the transition matrix.

113 Views Asked by At

Specify the classes of the following Markov chain, and determine whether they are transient or recurrent, then calculate $E[N_{i}]$ and then $f_i$ for all in state spaces of the Markov chain depicted by the transition matrix.

$\textbf{P} = \begin{Vmatrix} \frac{1}{4} & \frac{3}{4} & 0 & 0 & 0\\ \\ \frac{1}{2} & \frac{1}{2} & 0 & 0 & 0\\ \\ 0 & 0 & 1 & 0 & 0\\ \\ 0 & 0 & \frac{1}{3} & \frac{2}{3} & 0\\ \\ 1 & 0 & 0 & 0 & 0\\ \end{Vmatrix},$

This is what I get for the classes and if they were recurrent or transient:

$S_1$ = {0,1}, recurrent. $S_2$ = {2}, recurrent. $S_3$ = {3}, transient. $S_4$ = {4}, transient.

Counting over all time, the total number of visits to state i, given that X0 = i, is given by an infinite sequence of indicator rvs $$N_i = \sum_{n=0}^{\infty} I\{X_n = i|X_0 = i\}$$ and has a geometric distribution, $$P(N_i = n) = f^{n−1}_{i}(1 − f_i), n ≥ 1.$$

Here, for any state $i$ we let $f_i$ denote the probability that, starting in state $i$, the process will ever reenter state $i$.

However, I am having problems finding $E[N_{i}]$ and $f_i$, can someone help me?