Stochastic model over a gambling game

Question

Problem :

You have $1$$\$$ and you want to gather $10\$$ fast. For this reason, you decide to play a game with the following rules. On each round, the probability of winning is $0<p<1$, regardless of the outcome of the previous rounds. Before each round, you choose the amount you will bet. If you win, you get double your bet and if not, you lose the amount that you have bet. You have decided to bet all the money you have, if these are less than $5\$$, or in any other case, as much as you need to get to $10\$$.

(a) What is the probability of getting to $10\$$ ?

(b) What is the probability of reaching $10\$$, if you bet $1\$$ per round ?

(c) What is the expected time until you lose all your money or reach $10\$$ on each one of the cases above ?

Attempt - Discussion :

For part (a), solving the following Boundary Values Problem for the probability that we need to calculate $\mathbb{P}[T_{10} < T_{0} | X_0 = 1]$ yields the corresponding dynamical function $\Phi_{10,5}$ which is the answer to the question $$$$\begin{cases} h(x) = p \cdot h(2x), \; x = \{1,2,3,4\}\\ h(x) = p + (1-p)\cdot h(2x-10), \; x = \{5,6,7,8,9\} \\ h(0) = 0 \\ h(10) = 1\end{cases}$$$$

My question though is about part (b) and (c). For part (b), one could form the transition probability matrix and solve a new boundary values problem, but that would be an $11 \times 11$ matrix which would be very lengthy. As for part (c), I can't seem how to proceed.

I would appreciate any thorough help for (b) and (c).

Answer 1

There are a number of approaches to part (b) that either simplify or avoid entirely the process of solving an $11 \times 11$ system.

1. Since the equations are fairly simple, you can reorder then to relate $h(x+1)$ to $h(x)$ alone, which will then be easy to solve. We start with $h(0) = 0$ and $h(1) = \frac12 h(0) + \frac12 h(2) = \frac12 h(2)$. From there, $$h(2) = \frac12 h(1) + \frac12 h(3) \implies h(2) = \frac14 h(2) + \frac12 h(3) \implies h(2) = \frac23 h(3)$$ and $$h(3) = \frac12 h(2) + \frac12 h(4) \implies h(3) = \frac13 h(3) + \frac12 h(4) \implies h(3) = \frac34 h(4)$$ and so on until you get the pattern. Once you see the pattern, checking that it's true is easy enough.
2. You can show that the solution to (b) is the same as the solution to (a) by a coupling argument. Starting at $x$, let a "round of betting" consist of a series of 1-dollar bets that end at one of $\{0,2x\}$ when $x \le 5$ or at one of $\{2x-10, 10\}$ when $x>5$. The outcomes of a round of betting are equally likely by symmetry, and each round of betting in (b) corresponds to a single bet in (a).
3. You can show that the solution will be the same no matter which gambling strategy you try, by appealing to Wald's identity. The idea is that because the game is fair, your expected winnings at the end must equal your starting capital, so $$h(x) \cdot \$10 + (1-h(x)) \cdot \$0 = \$x$$ for all $x$, which tells us immediately what $h(x)$ must be. (But to make this rigorous, we need to know that the betting strategy is a stopping rule, as well as some finiteness condition.)

Part (c) is a question about hitting time and so we can also solve it by setting up a system of equations. For example, for the "careful" betting strategy limited to one dollar at every bet, we have $$ t(x) = 1 + \frac12 t(x+1) + \frac12 t(x-1), \qquad t(0) = t(10) = 0 $$ where $t(x)$ is the expected number of steps remaining from state $x$. But this requires a greater level of either patience or trickery to solve, and now it is no longer the case that all strategies give the same answer.