Expectation of a game where player doubles the bet every time he wins

Question

Player P plays a game whereby he bets on an event E occuring.

Event E occurs with probability q. So ¬E occurs with probability (1 - q)

The payoff per round is 1:1 -> For every 1 unit P bets on E , if E occurs, E collects 1 unit as the payoff, and keeps the 1 unit he bet; and if ¬E occurs, P loses the 1 unit he bet.

He adopts a strategy of playing agressive when he is winning, and playing safe when he is losing:

1. P plays exactly 10 rounds of the game
2. P always bets on E in every round
3. P will start the first round with a bet of 1 unit
4. After each and every loss, P will bet 1 unit on the next round, if there is a next round
5. If P wins the i^th round, he will increase his bet for the next round ( (i + 1)^th round ) by 2 times of the amount he bet in the i^th round.

ie, P bets 1 unit in i^th round and won. He will bet 2 units in the (i+1)^th round. If he wins again, he will bet 4 units in the (i+2)^th round.

6. After a triple consecutive win, P returns to betting 1 unit on the following round.
7. Rules 1 to 6 applies to every round of the game.

An example of a game,

Outcomes	E	E	E	E	¬E	¬E	E	E	¬E	¬E
Bets	1	2	4	1	2	1	1	2	4	1

Question: What is the expectation of the payoff if P adopts this strategy?

Note: No use of markov chain

##################################################

I need help with this question. I was thinking of using the usual way of calculating expectation, until I realised that the payoffs for games with the same number of wins and losses differs with the sequence of occurances of the wins and losses.

I tried to calculate the expectation of playing 1 round, 2 rounds, 3 rounds ... individually to try to find a relationship between them, but to no avail. I could not find any perceivable relation. In fact, the expectation became too tedious to calculate beyond 3 rounds.

Is there any intuitive way of solving this problem?

Answer 1

This solution was written before the poster modified the question and added rule 6. The following is only valid if P continues to double indefinitely after each win.

Let $E(n)$ be the expected payoff on the next $n$ rounds, assuming P stakes 1 unit on the first of those rounds. So we want $E(10)$.

The first thing to note is that with n rounds to play, if P stakes $x$ units on the first of those rounds, the expected payoff will be $xE(n)$ (the expectation scales up).

With $n+1$ rounds left to play, with P staking 1 unit on the first of those:

• with probability $q$, P will gain 1 and (because they will then stake 2) expect to gain $2E(n)$ on their remaining rounds
• with probability $1-q$, P will lose 1 and then expect to gain $E(n)$ on their remaining rounds.

So

$E(n+1) = q(1+2E(n)) + (1-q)(-1+E(n))$

$E(n+1) = 2q-1+(q+1)E(n)$

This is a recurrence relation that we can solve, using the initial condition that:

$E(1) = q - (1-q) = 2q - 1$

By considering a solution of the form $E(n) = A\alpha^n+B$ the recurrence relation leads to $\alpha=(q+1)$ and $B=\frac{1-2q}{q}$. The initial condition leads to $A = \frac{2q-1}{q}$. So

$E(n) = \frac{(2q-1)((q+1)^n-1)}{q}$

Substitute $n=10$ to answer the question.

Answer 2

Suppose we have $n$ rounds to go and P's stake is going to be $x$ on the first of those rounds. $x$ can be 1, 2 or 4. If it's 1 or 2 and they win they double their stake, but if it's 4 and they win, that signals three wins and their stake returns to 1.

Let $E(n,x)$ be the expected payout on the $n$ rounds assuming the stake of $x$.

The following three equations then follow (by considering the probability of a win then the probability of a loss):

$E(n+1,1) = q(1+E(n,2))+(1-q)(-1+E(n,1)) = 2q-1 + (1-q)E(n,1)+qE(n,2)$

$E(n+1,2) = q(1+E(n,4)) + (1-q)(-2+E(n,1))=2(2q-1)+(1-q)E(n,1)+qE(n,4)$

$E(n+1,4) = q(1+E(n,1)) + (1-q)(-4+E(n,1))=4(2q-1)+E(n,1)$

Initial conditions for these recurrence relations are:

$E(1,1) = q + (1-q)(-1) = 2q - 1$

Similarly, $E(1,2) = 2(2q-1)$

and $E(1,4) = 4(2q-1)$

If we express this using matrices, then:

Define $E_n = \begin{pmatrix} E(n,1) \\ E(n,2) \\ E(n,4) \\ \end{pmatrix}$

Define $u = (2q-1)\begin{pmatrix} 1 \\ 2 \\ 4 \\ \end{pmatrix}$

Define $Q = \begin{pmatrix} 1-q & q & 0 \\ 1-q & 0 & q \\ 1 & 0 & 0 \\ \end{pmatrix}$

Then $E_1 = u$

and $E_{n+1} = u + Q E_n$

The solution to this takes the form

$E_n=Q^n a + b$

where $b = (I-Q)^{-1} u$ and $a = Q^{-1} (u-b)$

Now all that remains is to calculate $Q^{-1}$ and $Q^{10}$, use those to calculate $a$ and $b$ and so calculate $E_{10}$ from which $E(10,1)$ can be deduced.