Stationary distribution for random walk on $\{1,2,...,N\}$

60 Views Asked by At

This is from Probability: An Introduction, by Grimmett and Welsh.

A random walk moves on the finite set $\{0, 1, 2, . . . , N\}$. When in the interior of the interval, it moves one step rightwards with probability $p$, or one step leftwards with probability $q$ (= $1 − p$). When it is at either endpoint, $0$ or $N$, and tries to leave the interval, it is retained at its current position. Assume $0 < p < 1$, and use the detailed balance equations to find the invariant distribution.

I've tried to use the detailed balance equations. Let $\pi$ be a stationary distribution and let $p_{ij}$ be the transition probability of moving from $i$ to $j$. Then:

$$ p \pi_i = q pi_{i+1}$$

The detailed balance equations hold where $\pi_i p_{ij} = \pi_j p_{ji}$ for $|j-1| \neq 1$, since both sides are zero. So I need to find $\pi$ such that it satisifies the displayed equation. I replace $\pi_i = \theta^i$ and obtain $\theta = p/q$.

I obtained that $\pi_i = A (p/q)^i$ where $A = (\sum_{i=1}^{N} (p/q)^i)^{-1} = \dfrac{1 - p/q}{p/q(1-(p/q)^n)}$.

Can someone let me know if this is correct? Thank you!

1

There are 1 best solutions below

0
On BEST ANSWER

Your equation would hold in case the local balance is satisfied; a priori it's not always the case.

You should get $\pi_i=\pi_{i-1}p+\pi_{i+1}q$ for $i=2,3,...,N-2$ and

$\pi_0=q\pi_0+q\pi_1$

$\pi_1=p\pi_0+q\pi_2$

$....$

$\pi_{N-1}=p\pi_{N-2}+q\pi_N$.

$\pi_N=p \pi_{N-1}+p\pi_N$

from the stationary distribution equation $\pi P=\pi$ in the matrix form; see e.g. Wikipedia.

Hence $\pi_1=p\pi_0+q\pi_2$

From the first two equations you get $\pi_0+\pi_1=\pi_0+q(\pi_1+\pi_2)$, i.e. $\pi_2=r\pi_1$ where $r=p/q$. Also $\pi_1=r \pi_0$ from the first equation.

Now re-iterating this and noting that $\pi_0+\pi_1+\dots+\pi_N=1$ gives you the answer.