Estimate expected payoff of rolling a dice, with choice of rolling up to $50$ times.

Question

This is an extended question of the classical rolling dice and give face value question.

You roll a dice, and you'll be paid by face value. If you're not satisfied, you can roll again. You are allowed $k$ rolls.

In the old question, if you are allowed two rolls, then the expected payoff is $E[\text{payoff}] = 4.25$.

If you are allowed $3$ rolls, the expected payoff is $E[\text{payoff}] = 4.67$.

If you can roll up to $50$ times, you can calculate the payoff using the formula and get $E = 5.999762$, notice that after $5^\text{th}$ roll, your expected payoff will be greater than $5$, so you'll only stop once you roll $6$.

---

So my question here is, without exact calculation(using geometric process), how would you estimate how many $9$s are there in the answer? Or another way to ask will be, is the expected payoff bigger than $5.9$? bigger than $5.99$? etc.

Answer 1

Let $a_n$ be the expected payoff of an $n$-roll game. We have $a_1=3.5$ and the recursion $$a_{n+1} = \frac{6 + \lceil a_n \rceil}{2} \cdot \frac{7 - \lceil a_n \rceil}{6} + a_n \cdot \frac{\lceil a_n \rceil - 1}{6}$$

You noted that for $n \ge 5$ we have $\lceil a_n \rceil = 6$, so the recursion in that case becomes $$a_{n+1} = 1 + a_n \cdot \frac{5}{6},\qquad n \ge 5.$$

Letting $p = 5/6$ we have we have the general formula \begin{align} a_n &= p^{n-5} a_5 + p^{n-6} + p^{n-7} + \cdots + p + 1 \\ &= p^{n-5} a_5 + \frac{1-p^{n-5}}{1-p} \\ &= (5/6)^{n-5} a_5 + 6(1-(5/6)^{n-5}) \\ &= 6 - (5/6)^{n-5} (6 - a_5) \end{align} for $n \ge 5$.

The second term $(5/6)^{n-5} (6 - a_5)$ tells you how far the expected payoff is from $6$; you can set this to $0.1$ or $0.01$ and solve for $n$ to answer your question.

Answer 2

Let $E_k$ be the expected payoff, if you're allowed to roll $k$ times, with the rules as you've described them. We can compute $E_k$ recursively.

---

With just $1$ roll, you must take what you get, since there are no more rolls. The expected value is therefore $$E_1 = \frac{1+2+3+4+5+6}{6} = 3.5$$

With $2$ rolls, if your first roll is $4$, $5$, or $6$, you will keep it, otherwise you will reroll and get $E_1$ from your next (and last) roll. Therefore, \begin{align*}E_2 &= \frac{4+5+6}{6}+\frac{1}{2}E_1 \\ &= 2.5+\frac{1}{2}(3.5) = 4.25\end{align*}

With $3$ rolls, if your first roll is $5$ or $6$, then you will keep it, otherwise you will reroll and get $E_2$ from your next two rolls. Therefore, \begin{align*} E_3 &= \frac{5+6}{6}+\frac{2}{3}E_2\\ &= \frac{11}{6}+\frac{2}{3}(4.25) = 4.\overline{6} \end{align*}

With $4$ rolls, if your first roll is $5$ or $6$, then you will keep it, otherwise you will reroll and get $E_3$ from your next three rolls. Therefore, \begin{align*} E_4 &= \frac{5+6}{6}+\frac{2}{3}E_3\\ &= \frac{11}{6}+\frac{2}{3}(4.\overline{6}) = 4.9\overline{4} \end{align*}

With $5$ rolls, if your first roll is $5$ or $6$, then you will keep it, otherwise you will reroll and get $E_4$ from your next three rolls. Therefore, \begin{align*} E_5 &= \frac{5+6}{6}+\frac{2}{3}E_4\\ &= \frac{11}{6}+\frac{2}{3}(4.9\overline{4}) = 5.1\overline{296} = \frac{277}{54} \end{align*}

Now, we have reached the point at which the recursion relation is stable. With more than $5$ rolls, you will always only keep the first roll if it is a $6$.

---

With $k$ rolls, $k>5$ if your first roll is $6$, you will keep it, otherwise you will reroll and get $E_{k-1}$ from the next $k-1$ rolls. Therefore,\begin{align*} E_k &= \frac{6}{6}+\frac{5}{6}E_{k-1}\\ E_k &= 1+\frac{5}{6}E_{k-1}\tag{1}\\\ \end{align*}

Notice that $$E_5 = \frac{277}{54} = 6 - \frac{47}{54}$$

The solution to the recurrence relation in $(1)$, with initial value $E_5 = 6- 47/54$, is:

$$E_k = 6 - \left(\frac{47 \cdot 144}{5^5}\left(\frac{5}{6}\right)^k\right)$$

---

Therefore, in general, the maximum expected payoff that you can achieve, when allowed $k$ rolls of a six-sided die, for any $k$, is $$\boxed{\,\,E_k \,=\,\begin{cases}7/2 \qquad &\text{if}\,\,\,k=1\phantom{l^{l^{l^{\overline{l}}}}}\\ 17/4 \qquad &\text{if}\,\,\,k=2\phantom{l^{l^{l^{\overline{l}}}}}\\ 14/3 \qquad &\text{if}\,\,\,k=3\phantom{l^{l^{l^{\overline{l}}}}}\\ 89/18 \qquad &\text{if}\,\,\,k=4\phantom{l^{l^{l^{\overline{l}}}}}\\\\6-\displaystyle\frac{6768}{3125}\left(\displaystyle\frac{5}{6}\right)^k \qquad &\text{if}\,\,\,k\geq 5\phantom{l_{l_{l_{l_l}}}}\\ \end{cases}\,\,\,}$$