Flip 4 coins, when you see a head, you get \$1. What is the expected return of this game? What if you can reflip (meaning reflip all coins) as many times as you want? What if you have to pay $1 each time to reflip? What is the best strategy?
For this problem, the EV of the first flip would be 2. So if I can choose to either reflip or keep the coins I'd choose to reflip if I get 0,1,2, if not then I'd keep the coins if they are 3,4. So the expected value for the first flip is $\frac{4}{16}(3) + \frac{1}{16}(4)=1$, and then for the second flip, we flip the four coins again to get an EV of 2. Multiplied by the respective probabilities, it's $1+ (2) \frac{11}{16} = 2.375$, which is better than the original 2.
But what happens when the game continues and you can reflip as many times as you want or if I need to pay \$1? Do I keep this strategy? I would assume if the game continues, then this strategy would work as well since I have a better chance of getting more than EV of first flips. But if I need to pay \$1 then I would have to accept a lower number the first flip (since $(1)\frac{5}{16} + (2) \frac{11}{16} < 2$). So say I accept 2,3,4 the first flip, I'd get $\frac{6}{16}(2) + \frac{4}{16}(3) + \frac{1}{16}(4)=1.75$ and reflip for 0 and 1 only. So $\frac{11}{16}(1.75)+\frac{5}{16}(1)=1.515 < 2$. So I go lower and reflip only when I get 0 heads. So the first flip EV is still 2 for $\frac{15}{16}$ of times and the second flip EV is 1 after paying 1 dollar, which is also worse than if I keep the flip the first round. So I keep the first flip.
Is this solution correct? I still don't know how to account for the scenario where I get infinite flips instead of 2.
You want to determine the maximum value $n \in \{0,1,2,3,4\}$ so that if you flip exactly $n$ heads, you do pay the $1\$$ to re-flip. This means that if you flip $(n+1)$ heads, you do not re-flip.
Let $E(n)$ denote your overall expected value as a function of $n$, where $n$ is as defined in the previous paragraph. You want to find all values of $n$ such that $E(n) \geq (n+1).$
This is because if you get $n$ heads, and decide to reflip, you are paying $1\$$ and forfeiting the $n\$$, for a total expenditure of $(n+1)\$$. You want to identify those values of $n$,where this expenditure is not greater than your expected value on a re-flip.
Consider $E(1).$
You are paying (in effect) $2\$$.
In exchange, $(5/16)$ of the time, you will be in the same position that you were in, $(6/16)$ of the time you get $2\$$, $(4/16)$ of the time you get $3\$$, and $(1/16)$ of the time you get $4\$$.
Therefore,
$$E(1) = (5/16)E(1) + (1/16)[25] \implies $$
$$(11/16)E(1) = (25/16) \implies E(1) > 2.$$
So, if you only get $1$ head, you should re-flip.
Consider $E(2).$
You are paying (in effect) $3\$$.
In exchange, $(11/16)$ of the time, you will be in the same position that you were in, $(4/16)$ of the time you get $3\$$, and $(1/16)$ of the time you get $4\$$.
Therefore,
$$E(2) = (11/16)E(2) + (1/16)[13] \implies $$
$$(5/16)E(2) = (13/16) \implies E(2) < 3.$$
So, if you only get $2$ heads, you should not re-flip.