Question. I have two urns and three balls: one red ball and two yellow balls. At the start of the experiment, all three of the balls are in one of the urns (the other urn is empty). Every minute, I pick one of the three balls at random (each is chosen with probability $\frac{1}{3}$), I take it out of the urn it is currently in, and I place it in the other urn. I stop (and the experiment ends) when both the yellow balls are in one urn and the red ball is in the other.
This experiment can be modelled as a Markov chain with six states (of which two are absorbing states). Write down what these six states are. (In other words, write down which physical situations each of the six states corresponds to.)
Draw the transition matrix.
Attempt. Consider the six states
- State 1. Urn 1 has one red ball and two yellow balls; urn 2 is empty.
- State 2. Urn 1 has one red ball and one yellow ball; urn 2 has one yellow ball.
- State 3. Urn 1 has two yellow balls; urn 2 has one red ball.
- State 4. Urn 1 has one yellow ball; urn 2 has one yellow and one red ball.
- State 5. Urn 1 is empty; urn 2 has one red and two yellow balls.
- State 6. Urn 1 has one red ball; urn 2 has two yellow balls.
This Markov chain has transition matrix
$$ \left( \begin{array}{cccccc} 0 & \frac{2}{3} & \frac{1}{3} & 0 & 0 & 0 \\ \frac{1}{3} & 0 & 0 & \frac{1}{3} & 0 & \frac{1}{3} \\ 0 & 0 & 1 & 0 & 0 & 0 \\ 0 & \frac{1}{3} & \frac{1}{3} & 0 & \frac{1}{3} & 0 \\ 0 & 0 & 0 & \frac{2}{3} & 0 & \frac{1}{3} \\ 0 & 0 & 0 & 0 & 0 & 1 \end{array} \right) $$
Pretty bulky question I do admit so apologies in advance.
But would you say this is right?
The states and Markov chain are correct. Visually, you can represent this as a random walk on the graph below.
Each state is represented by the contents of your favorite urn, so we start either in the top left or the bottom right corner. The edges with thick borders are absorbing states.
Corner vertices go horizontally $\frac23$ of the time and vertically $\frac13$ of the time, since yellow ball is more likely to be chosen. I was too lazy to put numbers on the Markov chain, so I've simply doubled some of the edges so that from each vertex, choosing an incident edge uniformly at random produces the correct transition probabilities.
You might also represent this as an $8$-state Markov chain if you distinguish the two yellow balls:
The advantage is that in this Markov chain, every state has $3$ adjacent states and all transition probabilities are $\frac13$.
Either way, I hope you aren't planning to solve the upcoming "what is the expected time until absorption" question by playing with a $6\times 6$ or $8\times 8$ matrix, because in this case there is a much more intuitive approach...