This was given as part of an assignment on dynamic programming using Bellman equations, but I feel like it could also be solved purely in terms of probability. (I've already submitted the assignment, just trying to see if the other way I came up with is correct).
Consider the following dice game:
The object of the game is to be the first player to reach 100 points. Each player’s turn consists of repeatedly rolling a die. After each roll, the player is faced with two choices: roll again, or hold (decline to roll again).
- If the player rolls a 1, the player scores 1, and it becomes the opponent’s turn.
- If the player rolls a number other than 1, the number is added to the player’s turn total, the sum of the rolls during the turn, and the player’s turn continues.
- If the player holds, the turn total is added to the player’s score, and it becomes the opponent’s turn.
For example, a couple of turns can go like this:
- Player A’s turn begins. A chooses to roll and rolls a 5. Turn total is 5. Then A decides to roll again and gets a 3. Turn total is 8. A chooses to roll again and gets a 6. With the turn total at 14 now, player A decides to hold. Now the turn total 14 is added to player A’s score, and the game moves to B’s turn.
- Player B’s turn begins. B chooses to roll and rolls a 4. Then B chooses to roll again and gets a 2. Turn total is 6. Decides to roll again and gets a 5. Turn total is 11. Chooses to roll again and gets 1. Now that B rolled a 1, the turn total is discarded, and simply a ‘1’ is added to player B’s score. The game goes to A’s turn again.
Here is what I have done:
At each roll, we can roll a 1 with probability $\frac{1}{6}$ and our turn end with this score, or roll higher than a 1 with probability $\frac{5}{6}$. In the later case, the expected value if we roll higher than a 1 is $(2+3+4+5+6)/5 = 4$
So we can think of the expected value of rolling $n$ times as this:
The expected value is thus:
\begin{align*} E(\text{1 roll}) & = \dfrac{1}{6}(1) + \dfrac{5}{6}(4)\\ E(\text{2 rolls}) & = \dfrac{1}{6}(1) + \dfrac{1}{6}\dfrac{5}{6}(1) + \left(\dfrac{5}{6}\right)^2(8)\\ E(\text{3 rolls}) & = \dfrac{1}{6}(1) + \dfrac{1}{6}\dfrac{5}{6}(1) + \dfrac{1}{6}\dfrac{5}{6}\dfrac{5}{6}(1) + \left(\dfrac{5}{6}\right)^3(12)\\ \vdots\\ E(n \text{ rolls}) &= \dfrac{1}{6}\sum_{k = 0}^{n-1}\left( \dfrac{5}{6}\right)^k + \left(\dfrac{5}{6}\right)^n(4n) \end{align*}
Computing this for different values of n:
This function reaches a maximum for six rolls, so I concluded that the optimal strategy is to always roll six times. Is this correct reasoning?