Maximizing Winnings - Dice Roll Strategy

Question

Let’s consider a simple dice game. Two fair 6-sided dice are rolled. Let $X$ is the sum of the two dice. If $X = 7$, then the game ends and you win nothing (winnings = 0). If $X \neq 7$, then you have the option of either stopping the game and receiving $X$ (what you rolled on your last roll) or starting the whole process over again.

Now consider this strategy to play: pick a number $i$, where $2 \leq i \leq 12$, and stop playing the first time that a value greater than or equal to $i$ is rolled (or until you are forced to stop as a result of rolling a 7). Define $Y_i = $ winnings when you use this strategy with chosen value $i$. We are interested in the value $i$ that maximizes the expected winnings $\mathbb{E}[Y_i]$ over the all possible choices of $i$. To make a long story short, it turns out that the value of $i$ that maximizes the expected winnings $\mathbb{E}[Y_i]$ for the game is $i = 8.$

For this problem, what we actually want is for you to explicitly compute the expected winnings $\mathbb{E}[Y_i]$ for $i = 5, 6, 8$ and $9$ to show why the expected winnings is maximized when $i = 8$. You do not need to consider the cases where $i = 2, 3, 4, 10, 11$ or $12$.

Attempt: Tried Expressing

$\mathbb{E}[Y_i \mid X = 7] = \text{-Winnings}$

$\mathbb{E}[Y_i \mid X < i, X \neq 7] = X + \mathbb{E}[Y_i]$

$\mathbb{E}[Y_i \mid X \geq i, X \neq 7] = X$

$\mathbb{E}[Y_i \mid X = 7] = -(\mathbb{E}[Y_i | X < i, X \neq 7] + \mathbb{E}[Y_i | X \geq i, X \neq 7])$

But no matter what I do, I'm getting an incorrect answer as $8$ should be the maximum, but it's not... Please help!

Answer 1

Let $p(n)$ be the probability of rolling a sum of $n$ on a single roll, so $$p(n)=\frac{1}{6}-\frac{|n-7|}{36}.$$ Then the expected value $e_i$ of the winnings while using the strategy that we stop only when we hit a sum of $i$ or greater is $$ e_i = \sum_{j=i,j \neq 7}^{12} p(j)\cdot j + \left(1-\frac{1}{6}-\sum_{j=i,j\neq 7}^{12} p(j) \right)e_i $$ Solving for $e_i$, we find $$e_2 = 35/6$$ $$e_3 = 208/35$$ $$e_4 = 202/33$$ $$e_5 = 19/3$$ $$e_6 = 85/13$$ $$e_7 = 20/3$$ $$e_8 = 20/3$$ $$e_9= 25/4$$ $$e_{10}=16/3$$ $$e_{11}=34/9$$ $$e_{12}=12/7$$ so the maximum occurs at $i=8$, which is the same as $i=7$.

Here is a python simulation for extra verification.

import math
import random

games = 10000000

strat= 2 # stop on strat or more
tots = 0

for i in range(games):
    done=0
    while(done=strat):
            done=1
    winning =0
    if (sum!=7):
        winning=sum

    tots = tots+winning

    if (i % 1000==0) and i>0:
        print i," ",tots*1./i

print tots*1./games

You can try it: output agrees with the exact calculated values above to very high precision.

Answer 2

As I said in my comments, I don't think the problem statement is complete, because it's not clear what your options are if you roll something other than seven on your first roll are. What would it mean to take the number you rolled on your last roll?

Leaving this problem aside for the moment, let's look at what happens if you choose to stop when you roll $9$ or better. The game continues until you roll one of the numbers $7,9,10,11,12.$ In the first case, you win nothing. In the second case, you win the expected value of the previous roll. Now, the previous roll must have been one of $2,3,4,5,6,8,$ so your expected winnings in this case are $E(X \mid X \in \{2,3,4,5,6,8\}.$ You have to weight this by the probability that the second case is the one that occurred, namely $P(X \ne 7 \mid X \in \{7, 9, 10, 11, 12\}).$

Similar reasoning applies if you choose $8,10,11,\text{ or }12$ as your target, of course.

This is dumb. Your decision should be based on the previous roll, not the current one.

Answer 3

So actually, your original conditioning approach is correct (using the law of total expectation). However, the forms of your equations are slightly off. What you want is: $$ E[Y_i] = E[Y_i|X<i, X\neq 7] p(X<i) + E[Y_i|X\geq i, X\neq 7] p(X>i) + 0 $$ So clearly $E[Y_i|X<i, X\neq 7] = E[Y_i]$ since you are rolling again. Meanwhile $E[Y_i|X\geq i, X\neq 7]p(X>i)$ is somewhat misleading. We are actually stopping for any X larger than or equal to i, so we have to average over all the stopping conditions (so we aren't computing a single $p(X>i)$). i.e. $$ \sum_{j\geq i}j p(j) $$

Let's see what we get for $i=8$, WE have to roll again for the sums (2,3,4,5,6), so that has a probability of 5/12 or 15/36. For the sums (8,9,10,11,12), we have to compute the average payout here over these sums (since we stop the game and take the payout). I get 35/9. So solving this equation:

$$ E[Y_i] = \frac{5}{12}E[Y_i] + 35/9 $$ Solving for this gives me 20/3. You can proceed similarly for any i.