Expected value of a die game where we roll adding the value of the roll to our points, but the running total of points exceeds $L$, we score $0$.

77 Views Asked by At

We are playing a game with a fair $N$-sided die whose faces are numbered $0$ through $N-1$ inclusive. We start with $0$ points. Each turn, we can either continue ($C$) or stop ($S$). If we choose stop, we roll one last time and add the result to our points; this final sum, whatever it may be, is our score for the game. If we choose continue, we roll and add the result to our points, but if this running sum exceeds $L$, the games end with a score of $0$; otherwise, we go to the next turn where we again choose to continue or stop.

Let $P \leq L$ be our current points and let

  • $E(P)$ be the expected score
  • $E_S(P)$ be the expected score if we stop; $E_S(P) = P + E_R$ where $E_R$ is the expected value from a roll, $\frac {N-1} 2$.
  • $E_C(P)$ be the expected score if we continue

then $E(P) = \max(E_S(P), E_C(P))$

$E_C(P)$ is tricky to get a handle on. I think it is

$$ E_C(P) = p \cdot E(P + \frac {L-P} 2) + (1-p) \cdot 0 $$

where $p$ is probability that $P + R_i \leq L$ where $R_i$ is the value of the roll we take if we continue. It is the probability that $R_i \leq L - P$, which is to say, $p = min(1, \frac {L-P} N)$. $\frac {L-P} 2$ comes from the fact that we're conditioning $R_i \leq L-P$, so the expected value of $R_i$ with that condition is $\frac {L-P} 2$. To summarize $p$, we have

$$ p = \begin{cases} 1 &\text{if } L-P \geq N\\ \frac {L-P} N &\text{if } 0 \leq L-P < N \end{cases} $$

So we have

$$ E(P) = \max(P + E_R,\ p \cdot E(P + \frac {L-P} 2)) $$

but I don't know how to solve this.


Or perhaps there is a way to find the value(s) of $P$ when $E_S(P) \geq E_C(P)$, so that once we reach it, we stop. While $L - P \geq N$, we should continue, since there is no risk of losing. For $0 \leq L - P < N$, we need to find $P$ s.t.

$$ \begin{align} P + E_R &\geq p \cdot E(P + \frac {L-P} 2)\\ P + \frac {N-1} 2 &\geq \frac {L-P} N \cdot E(P + \frac {L-P} 2) \end{align} $$

I have a gut feeling that the the optimal strategy is to continue until $P \geq L - \frac {N-1} 2$, since at that stage, if we continue we have a greater than or equal to $50$% chance of surpassing $L$ with the next roll and scoring $0$ points.

1

There are 1 best solutions below

0
On

When trying to get an insight into a game you are trying to design, it is best to start with the simplest analogous game, and add complications later.

So here, instead of a $0, \small(N-1)$ die, let us have a standard $D6$.
You can easily modify it back.

Suppose the limit for this game is $35$, then as long as we are at $23$ or below, it is obvious that we should continue, and we can work out the expected overall gain from here for each possible position from $24$ onwards. Note that in effect we have to look $2$ rolls ahead !

So from $24$, $E_s(P) = 24 + 3 = 27$
and $E_c(P) = \frac{35}{36}(24+\frac{240}{36}) \approx 29.81$

You can see that it won't be a simple computation encapsulated in one formula, eg if you are at $25$

$E_s(P) = 25+3 = 28$, which is simple, but

$E_c(P) = \frac{33}{36}(28+ \frac{218}{36}) \approx 31.21$

This is happening because with your condition, we have to think two rolls ahead with associated probabilities.