There are two boxes. The red box contains four red marbles and one blue marble, and the blue box contains five blue marbles and three red marbles. The following experiment is done a hundred times: A marble is randomly drawn from one of the boxes, its color recorded, then replaced in the same box. The first drawing is from the red box, but each subsequent drawing is determined by the color of the marble most recently drawn: if it is red, the next drawing comes from the red box; if it is blue, the next drawing comes from the blue box. What is the probability the hundredth marble is red?
From the question, I calculate
$$P(Red | Red)= \frac{4}{5}=0.8, P(Red | Blue)= \frac{1}{5}=0.2,...$$
I use the above probabilities to set up a transition matrix: \begin{bmatrix} 0.8 & 0.2 \\ 0.375 & 0.675 \end{bmatrix}
Since the question is asking for the hundredth marble, I know that I am looking for a steady state. In a steady state, the probability of red in next ball is the same as the probability of red in this ball. If $P(r)=Probability of Red$, then: $$\frac{4}{5}r+\frac{1}{5}(1-r)=r\implies r=\frac{15}{23}$$
This also matches what I found by raising the transition matrix to the twentieth power.
However, is there anything else I need to show? In particular, what is the general condition for a transition matrix to have a steady state? And how do I show that this matrix in particular will achieve a steady state.