Mean duration of the renewal intervals

214 Views Asked by At

$\textbf{Q}$: Let $\{X_n; n = 0, 1,...\}$ be a two-state Markov chain with the transition probability matrix

$$\begin{bmatrix} 1-a & a\\ b & 1-b \end{bmatrix}$$

where state $0$ represents an operating state of some system, while state 1 represents a repair state. We assume that the process begins in state $X_0 = 0$, and then the successive returns to state 0 from the repair state form a renewal process. Determine the mean duration of one of these renewal intervals.

$\textbf{Answer provided in the textbook}$: $\frac{1}{a}+\frac{1}{b}$

I am still new to this topic on renewal processes, and I am clueless as to how the above answer is derived, as stated in the book. I thought of deriving the above via the stationary distribution, but I am also clueless on that part. Am I on the right track? Some help will be great!

1

There are 1 best solutions below

5
On BEST ANSWER

Let $S_0 = \inf\{n>0:X_n=0,X_{n-1}=1\}$ and $S_{k+1} = \inf\{n>S_k:X_n=0,X_{n-1}=1\}$ for $k\geqslant 1$. Then $\{S_n:n\geqslant0\}$ is a renewal process. Let $X\sim\mathrm{Geo}(a)$ and $Y\sim\mathrm{Geo}(b)$ distribution, then $S_{k+1}-S_k\stackrel d=X+Y$, so \begin{align} \mathbb P(S_{k-1}-S_k=m) &= \mathbb P(X+Y=m)\\ &=\sum_{j=1}^{m-1} \mathbb P(X+Y=m, X=j)\\ &=\sum_{j=1}^{m-1} \mathbb P(X=j)\mathbb P(Y=m-j)\\ &=\sum_{j=1}^{m-1} a(1-a)^{j-1}b(1-b)^{m-1-j}\\ &=\frac{ab}{1-a}(1-b)^{m-1}\sum_{j=1}^{m-1}\left(\frac{1-a}{1-b}\right)^j\\ &=\frac{ab}{a-b}\left( (1-b)^{m-1}-(1-a)^{m-1}\right). \end{align} In particular, $$ \mathbb E[S_1-S_0] = \mathbb E[X] + \mathbb E[Y] = \frac1a + \frac1b. $$