How can i interpret this absorbing markov chain to solve a probability question?

Question

I try to solve a simple question; if I toss a coin and repeat it until a tails come up, what is the mean number of steps? (I want to solve another question but it is just a complicated version of this)

you know it, it finishes in 1 step with probability 1/2, and in 2 steps with p=1/4 and so on.. so answer is Sum (1/2)^n *n, n =1 to inf and it is 2

I tried to form an absorbing markov chain but i couldn't interpret it

\begin{bmatrix} 0&1/2&1/2&0 \\0&0&0&1 \\1&0&0&0\\0&0&0&1\end{bmatrix}.

(I couldnt write names on it) First is Start; Second is Tails; Third is Heads and last is End

I took upper 3x3 upper Q matrix and found Sum Q^k = (I-Q)^-1 and multiplied it with [1 1 1]'

Result is [4 1 5]'

So system is in Start 4 times, in Tails 1 and in Heads 5 times before exiting system.

What did I do wrong and How can i reach answer from this matrix? Note that I am not mathematician and I am now learning markov chains. Thanks in advance

Answer 1

Here's a summary of the answer:

Start from an initial state $O$, other states being $H$ and $T$, where $T$ is the absorbing state.

\begin{align*} \begin{pmatrix} & O & H & T \\ O & 0 & \frac{1}{2} & \frac{1}{2} \\ H & 0 & \frac{1}{2} & \frac{1}{2} \\ T & 0 & 0 & 1 \\ \end{pmatrix} \end{align*}

Next, compute:

\begin{align*} (I-Q)^{-1} = \left(\begin{array}{rr} 1 & 1 \\ 0 & 2 \end{array}\right) \end{align*}

Answer 2

No need for Markov chains. Let $X = $ # tosses until a tail comes up. $X$ takes values $k = 1, 2, 3, \dots$. Let $p$ be the probability of a coin coming up tails, and $1-p$ the probability of it coming up heads. If you are assuming the coin is fair, then $p = 1/2$. Assuming all flips are independent and identically distributed, \begin{align*} P(X = k) &= P\left(\{k^{th}\text{ flip is tails}\} \cap \cap_{i=1}^{k-1}\{i^{th}\text{ flip is heads}\} \right) \\ &= P(\{k^{th}\text{ flip is tails}\})\prod_{i=1}^{k-1}P(\{i^{th}\text{ flip is heads}\}) \tag{independence}\\ &= p(1-p)^{k-1}. \tag{identical distribution} \end{align*} Then it remains only to calculate the expectation of the random variable $X$. This turns out to be a geometric series. Indeed, $X$ is known as a "geometric random variable." You should find $E[X] = 1/p$.