The problem:
A man is playing two slot-machines. The first machine pays off with probability c the second with probability d. If he loses, he plays the same machine again. If he wins, he switches to the other machine. Let S, be the state of playing the i-th machine. Create the transition matrix and, assuming c=1/2 and d=1/4, find the fundamental matrix and intreprete it.
What I've done:
I found both matrices, but I don't know how to interprete the fundamental matrix, help would be appreciated.
$$T =\begin{bmatrix} 1-c & c \\ d & 1-d \end{bmatrix}$$ $$Z =\begin{bmatrix} 11/9 & -2/9 \\ -1/9 & 10/9 \end{bmatrix}$$
Where $T$ is the transition matrix and $Z$ is the fundamental matrix, calculated after replacing the given values for c and d.
The fundamental matrix of an ergodic Markov chain is
$$Z = \left(I-P+W\right)^{-1}$$
where $P$ is the transition matrix and $W$ is a matrix where each row is the fixed probability vector $w = (w_i)$ (the limiting distribution).
You can calculate the mean first passage times $m_{ij}$ (expected number of steps to reach state $j$ when starting from the state $i$, when $i\neq j$) from $Z=[z_{ij}]$ with the formula
$$m_{ij} = \frac{z_{jj}-z_{ij}}{w_j}$$
Source: this book, Theorem 11.16.
As you can, see those are $$m_{12} = \frac{\frac{10}{9}-(-\frac{2}{9})}{\frac{2}{3}} = 2$$ and $$m_{21} = \frac{\frac{11}{9}-(-\frac{1}{9})}{\frac{1}{3}} = 4$$
which we of course in this case see also straight from the probabilities as their inverses, since the number of steps we stay in each state are geometric random variables.