Attached above is an image from an exercice I have worked on, I have answered all questions but particularily struggled with question 4. When we were given the solutions, the correction simply said: $E_3(N(4)) = \frac{\rho_{34}}{1- \rho{44}}$ which equals infinity because the denominator is zero, but I struggle to understand how the expression was achieved. Is there a general formula for this? I couldn't find it in my class notes. How can we get the expected number of visits to class 3 starting at 2 , for instance?
If you start in state 3, then you enter state 4 with positive probability, and once you enter state 4, you stay there indefinitely. Therefore, you're expected to visit state 4 infinitely many times, if you start in state 3.