One-step probability transition matrix

Question

1. Start by rolling one die
2. If the outcome is even, roll two dice on the next turn
3. If odd, roll one die on the next turn
4. If two dice are rolled and sum is odd, roll one die next turn
5. If two dice are rolled and sum is even, roll two die next turn
6. Game ends, when a sum of 7 or 12 appears.

Write a one-step probability transition matrix for a Markov chain that can describe this situation.

I don't really get how to start on this question. I know the starting probability is 0.5 deciding even/odd.

Answer 1

We can identify $3$ states:

1. About to roll one die.$\\[4pt]$
2. About to roll two dice.$\\[4pt]$
3. Terminal state.

For the above states, the transition matrix is the $3{\times}3$ matrix whose $ij$-th entry $p_{ij}$ is the probability to transition from state $i$ to state $j$ in one turn.

For example \begin{align*} p_{11}&=\frac{1}{2}\\[4pt] p_{23}&=\frac{6}{36}+\frac{1}{36}=\frac{7}{36}\\[4pt] \end{align*} Can you work out the remaining $7$ transition probabilities?