I need to write an essay on the Game of Life board game, and so I studied up on Markov Chains to help me calculate the probabilities and average payoffs for the spaces; however I'm not sure whether I'm grasping the concept entriely, so I tried applying the concepts to a smaller and far simpler board game:
If I get a heads on a coin flip, I advance 1 space, but if I get a tails I advance 2. So $P(moving\;1\;space) = 0.5$ and $P(moving\;2\;spaces) = 0.5$. Space 6 is the finish so $p_{S6\:S6}=1$. If I make a transition matrix $P$ for the board game it should look like this:
$$ P = \begin{pmatrix} 0 & 0.5 & 0.5 & 0 & 0 & 0\\ 0 & 0 & 0.5 & 0.5 & 0 & 0\\ 0 & 0 & 0 & 0.5 & 0.5 & 0\\ 0 & 0 & 0 & 0 & 0.5 & 0.5\\ 0 & 0 & 0 & 0 & 0.5 & 0.5\\ 0 & 0 & 0 & 0 & 0 & 1 \end{pmatrix} $$
I then calulated the matrix $H$ where $h_{ij}$ gives the probability that a process will end up in transient state $j$ given that it starts in transient state $i$. To do this I used the formula $H = (N - I)N_{dg}^{-1}$ where $N$ is the fundamental matrix. This gave me the following:
$$ H = \begin{pmatrix} 0 &0.5 &0.75 &0.625 &0.6875 \\ 0 &0 &0.5 &0.75 &0.625 \\ 0 &0 &0 &0.5 &0.75 \\ 0 &0 &0 &0 &0.5 \\ 0 &0 &0 &0 &0.5 \end{pmatrix} $$
So if I wanted to know what the probability of landing of Space 3 was so I could calculate its average payoff, would I look at the (1, 3) entry of $H$? Because that would mean the probability of landing in space 3 given I started in space 1, and the player always starts in space 1.
So is the first row of $H$ the only row I need to focus on? Or is my thinking incorrect?
Yes, $(1,3)$ is the probability of landing on 3 having started at 1. And generally, if you're interested just in the probability of landing on a particular space at any point in the game, the first row is indeed all you need.