Can someone help me with this problem on Markov Chain?

81 Views Asked by At

There is a system which follows the equation: $X(n+1) = X(n) - 1 + A(n)$, where $A(n)$ is a random variable taking values $0,1,2$ with probabilities $p_0$, $p_1$ and $p_2$, respectively. Now, $X(n) = n$ and $X(n)\geq1$ are two provided conditions. I need to find the relation between $p_0$, $p_1$ and $p_2$ which ensures stability of the Markov Chain.

Answer is $p_0>p_2$, but I can't figure out the way to proceed.

Just for reference, this is actual a model of packet arrivals in a queueing network. Basically, $n$ is an index of discrete time, and $X(n) = n$ implies that the number of packets that arrived at time $n$ equals $n$ itself.

My progress: I tried to derive the steady state matrix, but am not getting $p_0>p_2$ as the answer. In fact, I don't understand how an inequality like this can be obtained.

1

There are 1 best solutions below

0
On

The transition probabilities are given by $$ P_{ij} =\begin{cases} p_0+p_1,& i=j=0\\ p_0,& j=i-1\\ p_1,& j=i\\ p_2,& j=i+1\\ 0,& \text{ otherwise}.\end{cases} $$ For $n=0,1,2\ldots$ let $$\nu(j) = \left(\frac{p_2}{p_0}\right)^j.$$ Then $\nu=\nu P$, so $\nu$ is an invariant measure for $P$. The system is stable if and only if $$\sum_{j=0}^\infty \nu(j)<\infty,$$ or $p_2<p_0$. In this case, $$\sum_{j=0}^\infty \nu(j) = \sum_{j=0}^\infty \left(\frac{p_2}{p_0}\right)^j = \frac{p_0}{p_0-p_2}.$$ Let $\pi = \left(1-\frac{p_2}{p_0}\right)\nu$, then $\sum_{j=0}^\infty \pi(j)=1$ and $\pi=\pi P$, so $\pi$ is the (unique) stationary distribution for $P$.