Trasition Matrix and Markov Chains

37 Views Asked by At

I have this old problem from a homework that I didn't understand quite well, but I have an exam coming up over the topic and was hoping to get some clarity

Consider the MC with this transition matrix

$$\left(\begin{array}[cccccc].6&.4&0&0&0&0\\ .2&.8&0&0&0&0\\ 0&0&.3&.6&.1&0\\ .1&.3&.2&0&.1&.3\\ 0&0&0&0&1&0\\ .2&0&.3&.15&.15&.2\end{array}\right)$$

a - Identify which states communicate with each other

b - Determine the period of each state

c - Determine which states are recurrent and transient

d - Beginning the chain at each transient state i, determine the expected number of steps until the chain is absorbed into either an absorbing state or a closed communicating class

e - Beginning the chain at each transient state, determine the probability that the chain will get absorbed by the closed communicating class

I believe states $1,2$ communicate, $3,4,6$ closed communicating, $5$ absorbing and that the periods re $[1,1,2,3,1,1]$. Lastly, all states except $5$ are transient but $5$ is reoccurring. I don't know if I got these wrong but I have no clue on how to solve d and e. Please help!