I encountered this problem which I find very contradictory to my intuition.
Could anyone enlighten me why my recursive formulation is wrong?
"Roll a fair dice until the game stops. The game stops when you get a 4, 5, or 6. For every 1, 2, or 3 your score increases by +1. If the game stops with a 4 or 5, you get paid the accumulated score. If the game stops with a 6 you get nothing. What is the expected payoff of this game?"
Here, my recursive formulation. Let $E$ denote the expected score. First three terms denote the event for 1,2,3 respectively, the fourth and the fifth term denote the event 4,5, the last term is for the event 6.
$$E = \frac{1}{6} \left( E + 1 \right) + \frac{1}{6} \left( E + 1 \right) + \frac{1}{6} \left( E + 1 \right) + \frac{1}{6} E + \frac{1}{6} E + \frac{1}{6}\cdot 0\\ = \frac{1}{2}\left( E + 1 \right) + \frac{1}{3} E$$
Solving for $E$, I get $E = 3$.
I don't want to take the Markov chain approach with infinite sums. I'd like to take the approach with intuitive recursive equation.
The solution guide I have takes a two-step approach of first calculating the expected number of tosses of $\{1,2,3\}$ before the game stops and then computes the expected score conditioning on the output $\{4,5\}$ versus $6$. This approach (while somewhat makes sense) ends up with $\frac{2}{3}$.
Could anyone please enlighten me what the problem in my recursive formulation is?
We condition on the result of the first toss. If that toss is $4$, $5$, or $6$, then the amount we get, and therefore the expectation, is $0$.
Given that the first toss is a $1$, $2$, or $3$, our expectation is not $E+1$. For the $1$ is not a sure thing. With probability $\frac{1}{3}$ it will not be credited to you, because the game will end with a $6$. It follows that $$E=\frac{1}{2}\left(E+\frac{2}{3}\right).$$ That yields $E=\frac{2}{3}$.
Remark: From the shape of your equation, it may be that you were trying to condition on the result of the last toss. I don't think that kind of conditioning argument can be pushed through. But if you were thinking in terms of the first toss, then your proposed conditioning argument, modified as described above, does work.
The only slight flaw with the conditioning argument is that it assumes that the expectation exists. That is intuitively clear, and can be justified by a series argument.