Exercise 1.3.2 of Norris, "Markov Chains"

Question

I'm working my way through Norris's classic textbook, but I'm having problems with this hitting probability question:

"A gambler has £2 and needs to increase it to £10 in a hurry. He can play a game with the following rules: a fair coin is tossed: if a player bets on the right side, he wins a sum equal to his stake, and his stake is returned; otherwise he loses his stake.

"The gambler decides to use a bold strategy in which he stakes all his money if he has £5 or less, and otherwise stakes just enough to increase his capital, if he wins, to £10. Let X_0=2 and let X_n be his capital after n throws.

"Prove that the gambler will achieve his aim with probability 1/5."

How does one solve this?

Answer 1

This looks to me as an absorbing Markov process. He starts at 2 GBP (forgive me for the denomination), if he wins he goes to 4, then to 8 and possibly to 10. But from 8 he can go back to 6 and from 6 back to 2 or wins to 10. From 2 he can also lose to 0. So we have the following "states": 0, 2, 4, 6, 8, and 10 where 0 and 10 are the absorbing states. From each state you can write down the probability according to the chances by the coin. Your matrix is a 6by6 matrix. If we partition this matrix in 4 sub matrices, we have a 2by2 matrix top left which is the Identity matrix (that's where the states "zero" and "10" are→game over). It's the top right matrix and bottom right matrix we need. According to the theories of Markov, the answer lies in the result of $R(I-Q)^-1$ where R is the top right matrix and Q the bottom right. The probability from the state "2" to "10" should give you the 1/5

Answer 2

Here is a martingale (not a markov chain) solution that comes from noticing that he's playing a fair game, i.e., if $X_n$ is his money at time $n$ then $E(X_{n+1}|X_n)=X_n$.

By the optional stopping theorem, the expected value when the game is finished is equal to the expected value when the game starts. He ends with either $\$0$ or $\$10$ (sorry, but I don't know how to make the pound symbol), and if $p$ is the probability that he ends in $\$10$, then $0(1-p)+10p=E(0)=2$, or $10p=2$.