Find the stationary distribution of Markov chain with reflecting barriers

134 Views Asked by At

(Feller Vol.1, P.424, Q.9) A chain with states $1,2, ... , \rho$ has a matrix whose first and last rows are $(q,p,0, ..., 0 )$ and $(0,...,0,q,p)$. In all other rows $p_{k, k+1} = p$, $p_{k, k-1} =q$. Find the stationary distribution.

I found that $u_k = p u_{k-1} + q u_{k+1}$ for $0<k<\rho$, and $u_0 = qu_0+pu_1$ and $u_\rho = qu_{\rho-1} + p u_{\rho}$. There are two particular solutions $(p/q)^k$ and $1$. So, the general solution is $u_k = A + B (p/q)^k$ for some constant $A$ and $B$. By rearranging the law two equations above, I can see that $u_0 = u_1$ and $u_\rho = u_{\rho-1}$. I think that using these as boundary conditions, I need to find two constants, but I am stuck here.

The answer for this problem is $u_r = (1- p/q)(p/q)^{r-1}(1-(p/q)^\rho)^{-1}$.

Am I on the right track? How can I get this answer?

Thank you in advance.

1

There are 1 best solutions below

0
On BEST ANSWER

I will approach the problem slightly different. We will find the stationary distribution using the Detailed Balance equation, hence $$\pi_i p_{i,j} = \pi_j p_{j,i} \, \ \forall i,j \in \{0,...,\rho\}$$

Where $\pi_i$ is the stationary distribution. Now, if we find $\pi_i$ that satisfies the Detailed Balance and $\sum_{i}\pi_i = 1$, then it is the stationary distribution. Note that $$ \pi_o p_{0,1} = \pi_1 p_{1,0} \iff \pi_0 \frac{p}{q} = \pi_1 $$ $$ \pi_1 p_{1,2} = \pi_2 p_{2,1} \iff \pi_2 = \pi_1 \frac{p}{q} = \pi_o \frac{p^2}{q^2} $$ $$ ... $$ $$ \pi_k = \pi_{k-1}\frac{p^k}{q^k} $$ Let's now use the condition that the sum is equal to 1.

$$ \sum_{k=0}^\rho \pi_k = \sum_{k=0}^\rho \pi_o \frac{p^k}{q^k} = 1 = \pi_o \sum_{k=0}^\rho \frac{p^k}{q^k} = \pi_o \frac{1-(\frac{p}{q})^\rho}{1-\frac{p}{q}} $$

We conclude that $$ \pi_k =\pi_o (\frac{p}{q})^k =(\frac{p}{q})^k \frac{1-\frac{p}{q}}{1-(\frac{p}{q})^\rho} $$

Note that I started from $0$ instead of $1$, so this is why I got $(\frac{p}{q})^k$ instead of $(\frac{p}{q})^{k-1}$.