Dice game - deciding whether to re-roll or not

Question

I am working on the following problem from a book:

A casino has a dice game. You can roll as many times as you want. For each roll you get paid $M$ dollars where $M$ is the number of dots on the roll as long as you do not roll a 6. The payment for each roll is additive. However, if you roll a 6, the game terminates and you lose your accumulated profit thus far. How much are you willing to spend on this game?

I am looking at the solution provided by the book, and I am confused. The solution is posted below. The part I am confused about is examining the threshhold for $n$ at which

$$ 5/6 \cdot n + 2.5 > n $$

Equality in the above expression holds when $n = 15$. I understand how they determined this solution, but it is not clear to me why this is the most optimal threshhold because the equation $5/6 * n + 2.5$ is derived from assuming you can only roll 1 more time. So if we have $n = \$16$, the solution is telling us that we shouldn't re-roll because the expectation of the profit of an additional roll is less than the current profit. But this assumes that we can only roll 1 additional time. Shouldn't be consider the cases of rolling more than 1 time if we have $n = \$16$ already?

Answer 1

Let $V(n)$ represent the expected number of dollars won if you have accumulated $n$ dollars. If you decide to stop, you win $n$ dollars. If you decide to roll, you will be in one of six states, each with probability $1/6$, and by conditioning on the value $r$ of the next roll, we obtain expected value $$\frac{1}{6} \sum_{r=1}^5 V(n+r) + \frac{1}{6}\cdot 0.$$ Hence $$V(n) = \max\left(n,\frac{1}{6} \sum_{r=1}^5 V(n+r)\right) \quad \text{for all $n \ge 0$} \tag1$$ If you can establish a boundary condition $V(n)=n$ for all $n\ge m$, then you can solve the recurrence for $n<m$ to find $V(0)$.

To find such a threshold $m$, note that the recurrence $(1)$ implies that $$m \ge \frac{1}{6} \sum_{r=1}^5 V(m+r) = \frac{1}{6} \sum_{r=1}^5 (m+r) = \frac{5m+15}{6},$$ so $m \ge 15$, but the book solution does not show that $m \le 15$. Indeed, if you set a larger threshold, say $m=16$, you still obtain $V(15)=15$. Implicit in the argument is that $V(n)=n$ implies $V(n+1)=n+1$. In words, if it is optimal to stop rolling in state $n$, then it is optimal to stop rolling in state $n+1$.

Answer 2

yes it makes sense to stop rolling after you have accumulated a certain no of dollars lets assume we have n dollars at a step then at the next step we have $\frac{5n}{6}\,+\,2.5$ dollars so this way if we continue at next $m^{th}$ step we have $(\frac{5^{m}}{6^{m}})\,(n-15)\,+\,15$ our objective would be to maximize this quantity we can clearly see that if we tend m to infinity that is we keep on playing the game then we have an expected outcome of 15 dollars no matter what we have at a particular step, so it makes no sense to start the game with more than 15 dollars at hand, at any step if we have >= 15 dollars then we check whether the expected outcome at the next step is greater than the current amount we have or not , if yes then we give another roll, if no then we stop.