A gambler bets on coin flips. With each flip, he wins $1$ dollar with probability $p$, and loses $1$ dollar with probability $1-p$. He starts with $2$ dollar and stops when he reaches either $0$ or $5$ dollar.
What is the probability that the gambler has $\$5$ in the end?
What is the expected number of coin flips he bets on?
Ok, so basically my assumption towards these question is that:
- Since $p$ and $1-p$ have been specified, its a biased coin.
- The steps are $0$ to $5$, and it starts from step $2$.
Could anyone please confirm or maybe write down your answers if this is correct? or if I have left out any answers could any experts please help me with this question? Your help is much appreciated.
The transition matrix for your absorbing Markov chain is
$$P= \begin{bmatrix} 0 & p & 0 &0&0& 1-p \\ 1-p & 0 & p & 0 & 0&0 \\ 0 & 1-p & 0 & p & 0&0 \\ 0 & 0 & 1-p & 0 & p & 0\\ 0 &0 & 0 & 0 & 1 & 0\\ 0 &0 & 0 & 0 & 0 & 1\\ \end{bmatrix} \begin{matrix} \dots$1\\ \dots$2\\ \dots$3\\ \dots$4\\ \dots$5\\ \dots$0\\ \end{matrix}$$
So
$$Q= \begin{bmatrix} 0 & p & 0 &0 \\ 1-p & 0 & p & 0 \\ 0 & 1-p & 0 & p \\ 0 & 0 & 1-p & 0 \\ \end{bmatrix} $$
$$R= \begin{bmatrix} 0 & 1-p \\ 0 & 0 \\ 0 & 0 \\ p & 0\\ \end{bmatrix} $$
and
$$ \begin{align} N&=(I-Q)^{-1}\\ &=\left( \begin{bmatrix} 1&0&0&0\\ 0&1&0&0\\ 0&0&1&0\\ 0&0&0&1\\ \end{bmatrix}- \begin{bmatrix} 0 & p & 0 &0 \\ 1-p & 0 & p & 0 \\ 0 & 1-p & 0 & p \\ 0 & 0 & 1-p & 0 \\ \end{bmatrix}\right)^{-1}\\ &=\left( \begin{bmatrix} 1 & -p & 0 &0 \\ p-1 & 1 & -p & 0 \\ 0 & p-1 & 1 & -p \\ 0 & 0 & p-1 & 1 \\ \end{bmatrix} \right)^{-1} \end{align} $$
You can invert this yourself because I'm too tired and lazy.
The point is that $B=NR$ will give you the answer to part 1
And $t=N\mathbf{1}$ will give you the expected number of steps for each starting position - you want the second row value.