Suppose Markov Chain has two states $1,2$ and transition probability $$P_{1,2}=q,\;P_{1,1}=1-q,\;P_{2,1}=p,\;P_{2,2}=1-p$$
Hence we have transition Matrix $$M=\begin{pmatrix}1-p & p\\q & 1-q\end{pmatrix}$$ At state $1$ payoff $2000$ and $\dfrac{-200}{1-p}$ at state 2
$$E[\text{payoff}]=\sum_{j=1}^\infty[1,0]M^j\left[2000,\frac{-200}{1-p}\right]^T=[1,0](I-M)^{-1}\left[2000,\frac{-200}{1-p}\right]^T$$
But $$M=\begin{pmatrix}p & p\\q & q\end{pmatrix}$$
which is not invertible?