Quant Trading interview probability question, what's wrong with my reasoning?

Question

I roll a die up to three times. You can decide to stop and choose the number on the die (where the number is your payoff) during each roll. What's your strategy?

I am confused about the strategy for the first roll. My reasoning is this: the chance of getting a 5 or 6 from rolls 2 and 3 is: $$1 - 4/6*4/6=1-4/9=5/9$$

So, you have a greater than $50\%$ chance to get a 5 or a 6 on either the 2nd or 3rd dice. Thus, you'd want to select the first dice only when it rolls a 6.

However, this answer is incorrect. The solution says the Expected value during the first roll is \$4.25, which I can understand how they computed, and thus the strategy would be to settle with either a 5 or 6 on the first roll, but I can't figure out my logical fallacy.

Answer 1

Work backwards from the end. The EV for the last roll is $3.5$. Thus on the second to last roll you should reroll if you get a $1,2,3$ and keep the die if you have a $4,5,6$. Thus we have an EV for the last two rolls of $\frac{1}{2}5 + \frac{1}{2}3.5 = 4.25$.

Hence for the first roll, you should keep the die if you get a $5$ or $6$, and reroll otherwise.

As for your fallacy, it's hard to say what your fallacy is because your reasoning is unclear. The goal is to maximize the EV, so it doesn't make sense to be talking about whether or not you have a greater than $50\%$ chance of rolling a $5$ or $6$ on the second and third rolls. That seems to be entirely unrelated to the question you're being asked. The question is how can you maximize your EV, which can be done by at each step comparing the EV of continuing to roll the die vs the value of keeping your current die.

Answer 2

There are multiple errors.

First you assume that your chance of getting $5$ or $6$ on the last two rolls is the same as the chance that the greater of two dice rolls will be $5$ or $6.$ This includes cases in which the first roll is a $4$ but the last roll is $5$ or $6.$ If you roll $4$ on roll 2, will you really discard that roll and take roll 3 as your final score? (Expected value $3.5$ on roll 3, with a $\frac12$ chance it is worse than the $4$ you got on roll 2 and only $\frac13$ chance that it is better.)

But if you stop after rolling $4$ on roll 2 (as you should), you won't roll a $5$ or $6$ on roll 3.

With optimal play after roll 2 (reroll on $3$ or lower, stay on a $4$ or higher), I figure the chance of getting a $5$ or $6$ on the last two rolls is just $\frac13 + \frac12\cdot\frac13 = \frac12 < \frac59.$ (You have a $\frac13$ chance to roll $5$ or $6$ on roll 2, and a $\frac12\cdot\frac13 = \frac16$ chance to roll less than $4$ on roll 2, reroll, and roll $5$ or $6$ on roll 3.)

But also consider that the chance of getting a score of $6$ on the last two rolls is only $\frac14$ (the same as getting a score of $5$). That's also the chance of getting a score of $4.$ and there is another $\frac14$ probability to end up with a score less than $4.$

So if your first roll is a $5$ and you decide to continue (and therefore discard that roll), you have only $\frac14$ probability to get a score higher than if you stop now, whereas you have a $\frac12$ probability to get a worse score. Moreover, the possible gain (if you improve) is at most one point, whereas the loss (if you do worse) is at least one point and very likely more.

In problems like this I don't see any substitute for actually working out what the ideal strategy is at each step using actual expected values, which means working backward (is in jgon's perfectly good answer) and doing the fully detailed probability calculations (not just rough lower bounds).