We are playing a game with a die where we roll and sum our rolls. We can choose to stop at any time and take the sum as our score, but if we roll the same face twice in a row (consecutively), we lose all our points. Calculate the range in which the maximal expected score lies?
My approach : Suppose the current sum so far is $S$, the last roll was $k$. Then expected score in next roll is $0 \times \frac{1}{6} + \left(S + \frac{1}{5}(1+2+\ldots+6-k) \right) \frac{5}{6} = \frac{5S}{6} + \frac{1}{6} (21-k)$, this needs to be greater than or equal to $S$ to make the next roll or equivalently $S \leq 21-k$ or $S+k \leq 21$
Thus, condition for making next roll is : current sum + last roll should be less than or equal to 21
This is my stopping strategy, now how do I compute expected score ?
Assume the game terminates with payoff $0$ if the value of a roll is equal to the value of the previous roll, otherwise, from "live" state $(s,k)$, the game terminates with payoff $s$ if $s+k > 21$.
Let $e(s,k)$ be the expected value of the game from live state $(s,k)$.
Then we have the recursion $$ e(s,k)=s \qquad\qquad\qquad\; $$ if $s+k > 21$, otherwise, if $s+k\le 21$, we have $$ e(s,k) = \sum_{j\in J_k} {\small{\frac{1}{6}}} e(s+j,j) $$ where $J_k=\{1,...,6\}{\setminus}\{k\}$.
Implementing the recursion in Maple we get $$ \qquad\;\;\;\;\; e(0,0) = \frac{678747596227}{78364164096} \approx 8.66 $$