Alex is taking a bioinformatics class and in each week he can be either up-to-date or he may have fallen behind. If he is up-to-date in a given week, the probability that he will be up-to-date (or behind) in the next week is 0.75 (or 0.25, respectively). If he is behind in the given week, the probability that he will be up-to-date (or behind) in the next week is 0.5 (or 0.5, respectively).
- Draw a Markov chain for this problem, showing the states and transition probabilities.
The question, that I am not able to answer, as I don't know how is
Assume Alex is up-to-date in the first class - what is the probability that he is up-to-date two classes later? What is the expected probability that he is behind after an infinitely long semester?
Give the transition probability matrix product for $$\lim_{x\to\infty} P^t$$
If Alex is up-to-date on day $1$ then there are exactly two paths that leave him up-to-date for day $3$: either he stays up-to-date through day $2$ or he falls behind for a day and then catches up. The probability for the first path is $\left(\frac 34 \right)^2$ and the probability for the second path is $\frac 14 \times \frac 12$. The total probability is just the sum of these.
For the "infinite sequence" question, let $p$ be the answer. So $p$ is the probability Alex is behind on a randomly selected day far in the future. Of course $p$ is also the probability that he's behind the next day. It follows that $$p=p\times \frac 12 + (1-p)\times \frac 14\implies p=\frac 13$$
Working along the same lines, you can compute the probability for all the transitions in the "infinite sequence" case.