Probability approach in the expected payoff of a dice game

Question

I am trying to understand the problem of expected payoff of a dice game explained here. I can roll the dice up to three times, but after each one I decide if I want to try once again or not. The idea is to find an optimal strategy that maximizes the expected payoff (expected number of spots; they do not sum up).

Let's say I know the optimal strategy: If in the first roll I get 1,2 or 3 I roll the dice once again. If in the second I get 1,2,3 or 4 I roll the dice once again. The end.

I wanted to calculate the expected number using probability $P(X=k)$ that I end up with $k$ spots. I am probably doing something wrong because I don't get the correct answer. My reasoning is the following:

$$P(X=\{1,2,3\}) = \underbrace{\frac{3}{6} \cdot \frac{4}{6} \cdot\frac{1}{6}}_{\text{3rd roll}}$$ $$P(X=4) = \underbrace{\frac{1}{6}}_{\text{1st roll}}+\underbrace{\frac{3}{6} \cdot \frac{4}{6} \cdot\frac{1}{6}}_{\text{3rd roll}}$$
$$P(X=5) = \underbrace{\frac{1}{6}}_{\text{1st roll}}+\underbrace{\frac{3}{6}\cdot\frac{1}{6}}_{\text{2nd roll}}+\underbrace{\frac{3}{6} \cdot\frac{4}{6}\cdot\frac{1}{6}}_{\text{3rd roll}}$$ $$P(X=6) = \frac{1}{6} + \frac{3}{6}\cdot\frac{1}{6}+\frac{3}{6} \cdot \frac{4}{6} \cdot\frac{1}{6}$$

Expected number is $E[X] = \sum_{k=1}^{6}kP(X=k) \approx 4.58$, which is not correct.

I appreciate any help.

Answer 1

Using your method:

$1$-roll game: $$E(X)=\sum_{k=1}^6 kP(X=k)=1\cdot \frac{1}{6}+2\cdot \frac16+3\cdot \frac16+4\cdot \frac16+5\cdot \frac16+6\cdot \frac16=3.5.$$ $2$-roll game:
$$E(X)=\sum_{k=1}^6 kP(X=k)=1\cdot \frac{3}{36}+2\cdot \frac{3}{36}+3\cdot \frac{3}{36}+\\ 4\cdot \left(\frac16+\frac{3}{36}\right)+5\cdot \left(\frac16+\frac{3}{36}\right)+6\cdot \left(\frac16+\frac{3}{36}\right)=4.25.$$ Note: The player rerolls if the first roll was $1,2,3$.

$3$-roll game: $$E(X)=\sum_{k=1}^6 kP(X=k)=1\cdot \frac{12}{216}+2\cdot \frac{12}{216}+3\cdot \frac{12}{216}+\\ 4\cdot \left(\frac{4}{36}+\frac{12}{216}\right)+5\cdot \left(\frac16+\frac{4}{36}+\frac{12}{216}\right)+6\cdot \left(\frac16+\frac{4}{36}+\frac{12}{216}\right)=4\frac23.$$ Note: The player rerolls if the first roll is $1,2,3,4$ and the second roll is $1,2,3$. You did the other way around!

Answer 2

I think you can approach this type of question inductively. Suppose you use an optimal stragedy for n rolls the expected value is $e_n$. Then for $n+1$ rolls, the expected value $e_{n+1}=\sum_{i: i\le e_n} \frac{1}{6} e_n+ \sum_{i: i>e_n} \frac{1}{6}(i)$. If your first roll is greater than $e_n$ then you dont roll anymore because you already get a value bigger than the expected optimal value for n rolls and if the value of first roll is smaller than $e_n$, then you are in the situation of n rolls and you keep performing the stragedy because the expected value for n rolls is higher than the firet roll.