Two players A and B toss two fair coins independently. Whoever gets the smaller number of heads will pay that many dollars to the other player.
For example, if player A tosses two coins and gets 2 heads, but player B tosses two coins and gets only one head, then player B will give one dollar to A.
Also, if player A tosses the two coins and gets no heads, while player B tosses and gets 2 heads, in that case player A will give 2 dollars to player B.
If both players have the same amount of heads in a certain round, no money change hands.
In addition, if the one who losses the round does not have enough money to pay, then that player will give whatever he has left.
The game ends when one player gets all the money.
Write the transition matrix for the Markov chain for amount of money held by player A if both players start with $5.00 each.
If we let $P_{i,j}$ be the likelihood of player starting with $i$ dollars, and ending with $j$ dollars after one move. Ignoring the edge cases for the moment, it should be clear that: $$P_{i,i+k} = P_{i,i-k}\ \forall k<i,\text{ further } P_{0,j} = P_{10,j} = 1\ \forall j$$
The second condition due to the fact that the game ends once player $A$ hits $\$10$ or $\$0$.
Let's for a moment assume that player has $2\leq i\leq8$ dollars. We know there to be 5 possible outcomes: Breakeven, win a dollar, lose a dollar, win two dollars, lose two dollars.
$$P_{i,i} = \text{Prob}((HH,HH),(TT,TT),(TH,TH),(TH,HT),(HT,TH),(HT,HT))$$ $$P_{i,i-1} = P_{i,i+1} = \text{Prob}((HH,HT),(HH,TH),(HT,TT),(TH,TT))$$ $$P_{i,i-2} = P_{i,i+2} = \text{Prob}((HH,TT))$$
Notice that each pair of double coinflips individually has likelihood of $2^{-4} = 16^{-1}$. Conveniently, if we add all events above they sum to $1$. Note that I'm double counting cases when they don't break even because there are two symmetric outcomes.
The issue arrises around the edges. If player A has $\$9$ for example, and rolls two more heads than player $B$ then he moves to node 10 and not the nonexistent 11th node.Because of this you need to include the likelihood of 2 Heads into the movement of node 9 to node 10.
Hope that's enough to get you started. If you find this somewhat confusing, imagine this game for the simpler case with 6 dollars and everyone starts with 3. Same structure, different scale and easier to play around with.
Also recall that the transition matrix does not change over time. Further, the transition matrix does not care at all what the initial conditions are. The fact that they ask for a transition matrix given the initial condition of $\$5$ is a red-herring designed to trick/distract