I am currently reading Introduction to Stochastic Models by Roe Goodman and have come across some unfamiliar notation for illustrating how Markov Chains work for this specific case of flipping a coin.
- It states that the coin shows heads with probability $p$, and I assume that $q$ represents $1 - p$ however this is causing some confusion. Is it assuming that we are starting at heads?
- The state space is either heads or tails, yes? What exactly does the progression of values from zero to three represent, is it the number of heads? That is what my current thought is as the $p$ arrows are to represent the probability of the coin landing on heads, but then this does not exactly hold up as the way $q$ is depicted it makes it seem as if it transitions back to heads with probability $q$.
Any help regarding the interpretations for this diagram is appreciated.
Based on the diagram, yes $q$ represents $1-p$. This is pretty standard notation.
The example is keeping track of the number of heads. So the state space is $\{0,1,2,...\}$. You should interpret the diagram as follows: Suppose we're at state $n$. One of two things happens. Either we flip heads, with probability $p$, and advance to state $n+1$. Or we obtain tails, with probability $q$, and stay at state $n$.