Two teams, A and B, are to play a best of seven series of games. Suppose that the outcomes of successive games are independent, and each is won by A with probability $p$ and won by B with probability $1 - p$. Let the state of the system be represented by the pair $(a, b)$, where $a$ is the number of games won by A, and $b$ is the number of games won by B. Specify the transition probability matrix. Note that $a + b\leq7$ and that the series ends whenever $a = 4$ or $b = 4$.
I've found the states $(0,0),(0,1),(0,2),(0,3),(0,4)$ $(1,0),(1,1),(1,2),(1,3),(1,4)$ $(2,0),(2,1),(2,2),(2,3),(2,4)$ $(3,0),(3,1),(3,2),(3,3),(3,4)$ $(4,0),(4,1),(4,2),(4,3)$
Is that states correct? How to find the transition probability matrix?
Yes, this list is correct.
For a typical state $(a, b)$, there are only two possible transitions, namely to $(a + 1, b)$, with probability $p$, and to $(a, b + 1)$, with probability $1 - p$. So, the column of the transition probability matrix corresponding to state $(a, b)$ only has two nonzero entries, corresponding to these states. By contrast, the states $(a, 4)$ and $(4, b)$ are absorbing.