Expected number of transition events to complete multiple synchronized Markov chains

419 Views Asked by At

Assume the expected number of transitions (events) it takes until a Markov chain with $G+1$ states ranging from $s=0$ to $s=G$ is completed is $M$. Suppose we have $K$ independent instances of this Markov chain synchronized such that a single event can lead to a transition to all of them. We are interested in the average number of transitions required to complete all of them.

One approach is to model this using a new Markov chain with a state space of size $(G+1)^K$ that corresponds to vectors (0,...0) to (G,...,G), calculate the transition probabilities and then derive the expected number of required transitions. We are already aware of how to do it. However, it is very interesting and insightful to approximate this using $M$ and $K$. Any suggestions is appreciated.

1

There are 1 best solutions below

1
On

Consider $l_i$ which is the expected number of moves required to reach node $G$ from node $i$.

Using this notation, we can create the following system of equations: $$l_i = 1 + \sum_{j}p_{i,j}l_j\text{ , where }p_{i,j}=P(s=i \to s=j)$$

In this instance $l_G=0$ and for $1\leq i< G$, $$l_i = 1 + (1-\alpha)l_{i} + \alpha l_{i+1}\implies l_i = \frac1{\alpha } + l_{i+1}\text{ for }0\leq i < G-1$$ Because of this, $l_{G-1} = \frac1{\alpha}, l_{G-2} = \frac2{\alpha},\ ...,\ l_0=\frac G{\alpha}$

Notice that we are dealing with the geometric distribution here. Since you have no likelihood of moving backwards and $\alpha$ is constant, $X = \min J > i\text{ s.t. }s_{i+J} = k+1\ |\ s_i = k \ \sim \text{Bin}(\alpha)$.

We then have that $$P(X = k) = \alpha(1-\alpha)^{k-1}\text{ and }E[X] = \sum_{k=1}^{\infty}\alpha(1-\alpha)^{k-1}k = \frac1\alpha$$

Since your Markov chain process is $G$ of i.i.d. $X_i\sim \text{Bin}(\alpha)$, $$Y = \sum_{i=1}^GX_i\sim\text{NegBin}(G, 1-\alpha)$$ Which is the negative binomial distribution. As we have demonstrated, $E[Y] = \frac G{\alpha}$, which agrees with the negative binomial definition.

Your final question about the expected number of moves for all chains to complete is a little more difficult. For this part of the problem we will require the theorem defining order statistics. Let $Y_{(i)}$ be the $i$th largest element in a sample set of size $K$ then the probability mass function is given by $$f_{Y_{(i)}}(y) = \frac{n!}{(i-1)!(n-i)!}F_{Y_{i}}(y)^{i-1}(1-F_{Y_{i}}(y))^{n-i}f_{Y_i}(y)$$

Unfortunately, the CDF of the Negative binomial distribution is not something I'm very familiar with (Regularized Incomplete Beta Function) so this is as far as I can take you. Notice that this is not a difficult problem to illustrate with simulation. Especially knowing that $Y_i$ is negative binomial.