Basic Die Game expected payout after re-roll

Question

Alice rolls a fair 6−sided die with the values 1−6 on the sides. She sees that value showing up and then is allowed to decide whether or not she wants to roll again. Each re-roll costs $1. Whenever she decides to stop, Alice receives a payout equal to the upface of the last die she rolled. Note that there is no limit on how many times Alice can re-roll. Assuming optimal play by Alice, what is her expected payout on this game

Answer 1

The following answer considers only strategies of the form: "stop after the $n$-th roll $X_n$ iff $X_n\geq t$", where $t$ is a threshold integer value.

Assume $0\leq t\leq 6$. Let $\tau=\inf \{n\geq 1:X_n \geq t\}$, so that $\tau$ has geometric distribution with parameter $(6-t+1)/6$ and note that the final payoff is $X_\tau -\tau +1$.

The expected final payoff is therefore $$E[X_\tau] -\frac{6}{6-t+1}+1.$$ Since $E[X_\tau] = E[X_1|X_1\geq t]= \frac{t+6}{2}$, it only remains to maximize $t\mapsto \frac{t+6}{2}-\frac{6}{6-t+1}+1$ over the set $\{1,\ldots,6\}$.

The maximal expected payoff is $4$, and it is reached for the threshold $t=3$. Thus the strategy is to stop rolling as soon as the roll is $\geq 3$, and keep rolling otherwise.