The probability to visit a state for the first time after n steps in a markov chain

Question

I have the following Markov Chain: (the probabilities are written above the arrows, and 'a' is a number between 0 and 1) I want to show that State-1 is a persistent state. To show that, I need to calculate the probability of visiting State-1 (starting from State-1) after n steps for every n up to infinity. I tried using the formula, but the transition matrix is not diagonalizable, so raising the matrix to high powers is difficult. I wonder if there is another way to do so (without raising the matrix to high powers) since the diagram is not so complicated. Thanks!

Answer 1

Persistence (or "recurrence") is a class property: any one state in a communicating class is recurrent iff every state in the class is recurrent (and then we can say that the class is recurrent).

Also, any finite closed communicating class is recurrent.

Your chain has a single class, which is finite and closed and therefore recurrent.

If you search for "markov chain closed class recurrent" you will find tons of references for all this stuff. See for example the book by James Norris, specifically Theorems 1.5.4 and 1.5.6 in Section 1.5.