In each turn, a player advances one space according to the arrows. (In general, the advancement from one space to another is deterministic. But when a player is on the green or red space, the player advances according to the probabilities shown) Suppose that a player collects a green token every time he or she lands on the green space and a red token every time he or she lands on the red space.
(a) On average, what fraction of a player’s tokens is green?
(b) What is the long-run rate that a player collects tokens (tokens / turn)?
(c) What fraction of turns, on average, does a player spend in the inner loop?
(d) Determine the steady-state probabilities $\pi_{n}$ (the fraction of time a player is in state n)
My attempt:
(a) I think there is a trick, by observing that $P(\text{next token is green|current token = red}) = 0.1$, while $P(\text{next token is green|current token = gree}) = 0.5$. But I am not sure if the fraction of this player's token is green is simply equal to $0.5+0.1=0.6$.
(b) I would need to use Renewal Reward theorem here, and I defined a cycle $=$ the next token obtained, given the current token was obtained. So the expected length of this cycle would be $4\times 0.6+11*1.4$. And the expected number of token per cycle = $1$, as it does not matter how we move, we would get a token at the end. Thus, the long-run rate = $\frac{1}{17.8} = \fbox{17.8}$. Is this correct?
(c) I am thinking of letting $s_{ij} = E(N^{j}$|start in state i) = expected number of visits to state $j$ given starting in state $i$. So I would need to compute $s_{iA} = \sum_{j=1}^{11} s_{jA}$ where $j = $ one of the remaining states. This means solving the $11\times 11$ system of equations, which is quite painful:( Is there any other way besides solving for $s_{iA}$ in this part?
(d) Well, again, solve for $\pi\ P = \pi$ with $\sum_{i=1}^{11} \pi_i = 1$, which is an $11\times 11$ system of equations. How could I solve this without inverting the matrix $P-I$, or Gaussian Elimination?
My question: Could anyone please help me out, as I got stuck on this problem for a while? Any help/thought would really be appreciated.