What's the optimal strategy of this dice game?

Question

I'm working on dynamic programming problems this week. The following is a dice game. I'm trying to find the optimal strategy using dynamic programming.

The game's description :

The player rolls two dice.

• If the sum of the numbers shown by the two dice are equal to the precedent sum (precedent round) then the player will loose all his reward.

• Otherwise the player will add the sum given ($2+3$ for example) to the cumulative rewards he has.

I started modeling the problem. The state of the system is as follow: I chose one random number (400 for example) that may be the maximum reward for $N$ rounds game. The state is $V_k(S,D)$ where $S$ is the cumulative sum and $D$ is the sum of the two numbers shown on the two dice.

If we suppose that the numbers of rounds is finite, what will be the optimal strategy in a simple form?

In the case of infinite game what's the mean of the reward?

Thank you in advance guys :)

Answer 1

Here's a Sage session that calculates the expected value and the optimal strategy:

sage: @CachedFunction
sage: def f(s,d):
...       if (s > 500):
...           return s    
...       exp = 0
...       for i in range (2,13):
...           if i != d:
...               p = (6 - abs (7 - i)) / 36
...               exp = exp + p * f (s + i,i)
...       return max (s,exp)    
sage: f (0,0)
5102856371420701910406444770125748115514599426793322429508600411301424025583837413991140684712401545/200826102839256313932089349950528975334638448502151718250240746889056739809428118955117185815543808
sage: _.n ()
25.4093282659829
sage: for i in range (2,13):
...       for s in range (0,500):
...           if f(s,i) == s:
...               print "If you rolled",i,", stop if you have at least",s
...               break
If you rolled 2 , stop if you have at least 250
If you rolled 3 , stop if you have at least 127
If you rolled 4 , stop if you have at least 86
If you rolled 5 , stop if you have at least 66
If you rolled 6 , stop if you have at least 54
If you rolled 7 , stop if you have at least 46
If you rolled 8 , stop if you have at least 53
If you rolled 9 , stop if you have at least 63
If you rolled 10 , stop if you have at least 81
If you rolled 11 , stop if you have at least 119
If you rolled 12 , stop if you have at least 241

Thus the game's value is about $25.4$, and the reward at which to stop depends strongly on the probability of repeating the last roll, and is also slightly higher for the lower of two equiprobable sums (e.g. $2$ and $12$), as might be expected.