Consider a birth and death process with infinitesimal parameter $\lambda_n$, $\mu_n$. Then the expected length of time for reaching state $r + 1$ starting from state 0 is $$\sum_{n=0}^r \frac{1}{\lambda_n\pi_n} \sum_{k=0}^n \pi_k$$
For the definition $\pi_n$ see the following: $$ \sum_{n=0}^{\infty} \pi_n \sum_{k=0}^n \frac{1}{\lambda_n\pi_n} = \infty$$
where $\pi_0=1$ and $\pi_n= \frac{\lambda_0\lambda_1\cdots \lambda_{n-1}}{\mu_0\mu_1\cdots\mu_{n-1}}$, $n=1,2,\ldots$.
In most practical examples of birth and death processes the last condition is met and the birth and death process associated with the prescribed parameters is uniquely determined.
Well, this is a very complicated exercise for me, first I tried to let $T^{\ast}_n$ denote the elapsed time of first entering state $n + 1$ starting from state $n$.
And then trying to derive a recursion relation for $E[T^{\ast}_n]$ but I can not solve it, I stuck trying to derive this recursion relation....
Could someone help me with hints, suggestions to solve this... Thanks for your time and help everyone.
For consistency of notation, define $\mu_0=0$. (Normally in birth-death processes $\mu_0$ is just not defined.)
Now the expected time of a transition from $i$ is $(\lambda_i+\mu_i)^{-1}$, the probability of the first transition from $i$ being a birth is $\frac{\lambda_i}{\lambda_i+\mu_i}$, and the probability of the first transition from $i$ being a death is $\frac{\mu_i}{\lambda_i+\mu_i}$. Therefore the expected time to hit $r+1$ started from $i$ satisfies
$$u(i)=(\lambda_i+\mu_i)^{-1} \left ( 1 + \lambda_i u(i+1) + \mu_i u(i-1) \right ),0 \leq i \leq r \\ u(r+1)=0.$$
(Here again for consistency of notation we arbitrarily define $u(-1)$; this has no effect because it is multiplied by zero anyway.) This is a tridiagonal linear system. Because it has Dirichlet boundary conditions ($u(-1)$ and $u(r+1)$ are given), you can solve it recursively instead of by elimination: plug in $i=r$ and use the boundary condition to get a relation between $u(r)$ and $u(r-1)$. Plug in this relation at $i=r-1$, to get a relation between $u(r-1)$ and $u(r-2)$, and so on. When you get to $i=0$ you can actually solve for $u(0)$, because $\mu_0=0$. This is actually what you want in this case, but if it weren't you could back-substitute to find the others.
This kind of procedure is the subject of renewal theory.