Stationary distribution of MC

312 Views Asked by At

A markov chain on states 0,1,... has transition probabilities $P_{ij}=\frac{1}{i+2}$ for $j=0,1,...,i,i+1$. Find the stationary distribution.

My attempt:

First I calculate the equations

$\Pi_0$=$\Pi_0/2+\Pi_1/3+\Pi_2/4+\Pi_3/5+...$ and so on

But then I don't know how to continue, I mean how to solve for $\Pi_k$.

Can somebody help me please?

1

There are 1 best solutions below

5
On BEST ANSWER

Well, instead of saying "and so on" write them down. You will see that they are all very similar to one another. For instance we have $$ \Pi_0 = \frac{\Pi_0}{2}+\frac{\Pi_1}{3}+ \frac{\Pi_2}{4}+\ldots\\\Pi_1 = \frac{\Pi_0}{2}+\frac{\Pi_1}{3}+ \frac{\Pi_2}{4}+\ldots\\\Pi_2 = \frac{\Pi_1}{3}+\frac{\Pi_2}{4}+ \frac{\Pi_3}{5}+\ldots$$

So we have $\Pi_0 = \Pi_1$ and $\Pi_1 = \frac{\Pi_0}{2}+\Pi_2.$ So immediately we have $\Pi_2= \Pi_0/2.$ Now we can write $\Pi_2 = \Pi_1/3+\Pi_3$ so that $\Pi_3 = \Pi_0/6.$ Then $\Pi_3 = \Pi_2/4 + \Pi_4$ so that $\Pi_4 = \Pi_0/24.$ See the pattern?

To see the factorial come out more clearly, try adding the equations together. If you add the equations from $\Pi_{i+1}$ on you get $$ \Pi_{i+1}+\Pi_{i+2}+\ldots = \frac{\Pi_i}{i+1} + 2\frac{\Pi_{i+1}}{i+2} +3\frac{\Pi_{i+2}}{i+3}+\ldots$$ which rearranges to $$ \frac{\Pi_{i}}{i+1} = i\left(\frac{\Pi_{i+1}}{i+2} +\frac{\Pi_{i+2}}{i+3}+\ldots\right)$$ and can be further rewritten to $$ \Pi_i = i\left(\frac{\Pi_i}{i+1}+\frac{\Pi_{i+1}}{i+2} +\frac{\Pi_{i+2}}{i+3}+\ldots\right)$$ But we can plug in $i-1$ for $i$ in the first equation to give $$ \frac{\Pi_{i-1}}{i} = (i-1)\left(\frac{\Pi_i}{i+1}+\frac{\Pi_{i+1}}{i+2} +\frac{\Pi_{i+2}}{i+3}+\ldots\right) = (i-1)\frac{\Pi_i}{i}$$ where we plugged in the second equation. This simplifies to $$ \Pi_i = \frac{\Pi_{i-1}}{i-1}.$$

(There might be an easier way to get this 1st order recursion equation that I'm missing...)