A gambler plays the following game: A fair coin is tossed until getting three times continuously head. When that happens the Gambler gets 20$\$$. Each round costs the gambler 1$ (even if he won the round). I want to figure out:
- Is that a fair game?
- What is the expected value for gambler's profit (or loss)?
- Suppose he can stop anytime can he increase the expected value for profit?
I thought solving that as a Markov chain by first defying $T_n$ to be the result of the nth toss and $$X_n=\cases{17&$T_{n-2}=T_{n-1}=T_n=$head\\-3&otherwise}$$ and then $$E[X_n]=\frac {17} 8-\frac {3\cdot 7} 8=-\frac 1 2$$which is negative and that seems a bit weird since Expected value is always non negative. Now if we sum over every $3\mid n$ (by threesomes) we get $E[\sum X_n]$ diverges to negative infinity.
How should I define the variables in the Markov Chain s.t I'll be able to solve the first two problems?
About the third, I'll be glad for a hint (intuitively he cannot but I can't explain why).
The expected value can certainly be negative. For example, suppose our game is you flip a coin. Heads no money changes hands, tails you pay me a dollar. The expected value to you is then $-50$ cents.
In your problem, the states are the number of heads since the last tails. Call them A, tails was the last flip, B, one heads since the last tails, C, two heads, and D, three heads. D is absorbing because the gambler wins. You start at A. Call $a,b,c$ the expected number of flips to get to D from the corresponding state. You should be able to write three equations in three unknowns for these and solve them.