Stationary distribution for a Markov chain which is not irreducible

727 Views Asked by At

I have a Markov chain with $K$ states $S$: {$s_1,s_2,...,s_K$}.

$s_1$ is reachable from any state in $S$; however not all the states can be reached from $s_1$.

What does the stationary distribution of this Markov chain look like? Does the solution of the function: $$\pi~P=\pi,$$ get the right answer?

Actually, I want to get the expectation of the states by: $$E=\sum_{s_i\in~S}{\pi_i*s_i}.$$

How can I calculate it correctly?

1

There are 1 best solutions below

2
On BEST ANSWER

Every stationary distribution $\pi$ solves $\pi P=\pi$ and puts mass zero on the states which $s_1$ does not communicate with. Thus the sum defining the expectation $E$ can omit these states.