Exercise on Markov chains

198 Views Asked by At

I'm preparing my Probability exam and I'm having trouble with exercise 2 here. The question is to consider the random walk on $E$ with transition matrix $p$ and find the communication classes (or irreducibility classes, or whatever they are called), classify the states based on recurrence and transience, show that 1 and 5 are aperiodic states, and calculate the probability for the chain to reach 1 starting from 2. I've done everything easily except for the last point. It is clear to me that I could calculate that probability as the sum over $n\in\mathbb{N}$ of the probabilities of reaching 1 from 2 in $n$ steps, but that sequence doesn't seem to have any particular properties, and calculating all terms is impossible, and even calculating a few becomes difficult in that I must consider all possible path and am sure to miss one at some point. I was thinking that the probability may be 1 since 1 is recurrent, but the equivalent condition for recurrence is that the probability of returning to the state starting from it is one, not that the probability of reaching the state from another one is 1. Can someone help me here?