We consider the following transition matrix for a markov chain with state space {A,B,C,D,E} :
$P= \left( \begin{array}{ccccc} \frac{1}{2} & 0 & 0 &0 &\frac{1}{2} \\ 0 & \frac{1}{2} & 0 &\frac{1}{2} &0 \\ 0 & 0 & 1 & 0 &0\\ 0 &\frac{1}{4} &\frac{1}{4} &\frac{1}{4} &\frac{1}{4}\\ \frac{1}{2} &0 &0 &0 &\frac{1}{2}\end{array} \right) $
(a) find its stationary distrbutions.
(b) Suppose we start in state B. In the long run, what is the probability that we are in state A at any given time?
Attempt: We observe here that A,C,E are recurrent states and B,D are transient. I have found out the stationary distributions to be $(0,0,1,0,0)$ and $(\frac{1}{2}, 0,0,0,\frac{1}{2})$
How do we do the second part?
Thank you for your time and help.
You have two ergodic SCC: $\{A,E\}$ and $\{C\}$. The probability you are looking for is the probability to reach $\{A,E\}$. This can be computed via absorbing Markov Chains:
http://en.wikipedia.org/wiki/Absorbing_Markov_chain
To sum up the wikipedia page, you have to consider the 2*2 matrix $Q$ of transitions between transient states ($B$ and $D$) and the 2*2 matrix $R$ that gives the transition from B and D to components $\{A,E\}$ and $\{C\}$. You get $Q=\left( \begin{array}{cc} \frac{1}{2}& \frac{1}{2} \\ \frac{1}{4} &\frac{1}{4} \end{array}\right)$
and $R= \left( \begin{array}{cc} 0& 0 \\ \frac{1}{4} &\frac{1}{4}\end{array} \right)$.
Then you compute the matrix $X=(I-Q)^{-1}R$, and the entry (1,1) is the probability you are looking for: reaching $\{A,E\}$ from $B$ (section "Absorbing probabilities" on wikipedia).
Notice that here, no computation is necessary, as the matrix $R$ is unchanged by swapping columns, you will get probability $\frac{1}{2}$ for each component.