Given the transition probability matrix with the order of column = row = $(1,2,3)$: $$P =\pmatrix{0.2 & 0.8 & 0\\ 0.4 & 0 & 0.6\\ 1 & 0 & 0}$$ representing a DTMC. Suppose that you get a reward of $5$ each time you are in state $1$, a reward of $7$ each time you are in state $2$, and a reward of $9$ each time you are in state $3$.
(a) Using renewal theory, determine the long-run reward rate (per time step).
(b) Using the theory of Markov chains, determine the long-run reward rate (per time step)
My attempt: (a) I tried to compute the expected time between visits to each state by first computing the fraction of time in each state through the equation: $\pi P = \pi$ with $\sum_{i=1}^{3} \pi_i = 1$. Solving this equation we get: $\pi_1 = 0.44$, $\pi_2 = 0.35$ and $\pi_3 = 0.21$. Now, I want to use the formula: Expected time between visits to state $i =$ $\frac{1}{v_i\pi_i}$, but then I could not compute $v_i$ from the given information. So I'm completely stuck here:(
My question: Could someone please help me find the instantaneous rate out of state $i$ from the given information? Or more directly, a right way compute expected time between visits to state $i$? Any thoughts would really be appreciated.