Hitting time of absorption state in a specific continuous time Markov chain

251 Views Asked by At

Let $(X_t)_{t\geq 0}$ be a CTMC on state-space $S=\{0,1,2,\dotsc\}$ and with $X_0=1$ a.s. and transitions $q_{i i+1}=i\beta$ for each $i\geq 1$ and $q_{i0}=\alpha/i$ for each $i \geq 1$. There are no other transitions and $q_{ii}=-(\frac{\alpha}i +i\beta)$ for $i \geq 1$ while $q_{00}=0$, so that $0$ is absorbing. Note $-q_{ii}>0$ for all $i \geq 1$.

We want to find the expected time to hit $0$ starting from $1$. That is, if $T_0=\inf\{t\geq0 : X_t =0\}$, we want to compute $k_1 :=\mathbb{E}_1(T_0):=\mathbb{E}(T_0 | X_0=1)$. The standard Markov chain theory says that the hitting time of $0$ starting from $n$ is the minimal non-negative solution of the system $$k_n = \frac{1}{(\frac{\alpha}n +n\beta)}+\frac{n\beta}{(\frac{\alpha}n +n\beta)} k_{n+1},\quad n \geq 1$$ with initial condition $k_0=0$. Rearranging this and using simplified notation, we have $$k_{n+1}=a_n k_n +b_n, \quad n \geq 1$$ and $k_n=0$ where $a_n = (\alpha+n^2 \beta)/(n^2 \beta)$ and $b_n = -1/(n\beta)$, so a first-order non-homogeneous recurrence with variable coefficients.

So, my question: Is there a "nicer" method to solve this specific first-order non-homogeneous recurrences with variable coefficients or is there an easier way to compute the expected hitting time $\mathbb{E}(T_0|X_0=1)$ of $0$?


According to wikipedia there is a "nice" solution method, where in our variable labeling we would obtain, $$k_n=\prod_{i=1}^{n-1} a_i \left(A_1 + \sum_{i=1}^{n-1} \frac{b_m}{\prod_{i=1}^m a_i}\right).$$ My attempt to simplify this in the case above, yielded $$k_n = C_{n-1} \left[A_1-\left(\frac{1}{\beta C_1}+\frac1{2\beta C_2}+\dotsc +\frac1{(n-1)\beta C_{n-1}}\right)\right],$$ where $$C_m = \prod_{i=1}^m a_i = \frac{(\alpha+\beta)(\alpha+2^2\beta) \cdot \dotsc \cdot (\alpha+m^2 \beta)}{1\cdot 2^2 \cdot \dotsc \cdot m^2 \beta^m},$$ from plugging in $a_i = (\alpha+i^2 \beta)/(i^2 \beta)$. Obviously, there's a little simplification if you multiply $C_{n-1}$ through but it does not seem too fruitful. I will certainly have to use the non-negative minimal solution property as well, but I do not see anywhere to apply it yet.

1

There are 1 best solutions below

1
On BEST ANSWER

There is no finite non-negative solution. Let $A_n = {\prod_{i = 1}^n a_i}$. The general solution of $k_{n + 1} = a_n k_n + b_n$ is $$k_n = A_{n - 1} \left( C + \sum_{i = 1}^{n - 1} \frac {b_i} {A_i} \right)$$ for $n \geq 1$ (there is no initial condition, the recurrence relation is not valid for $n = 0$). The limit of $A_n$ is finite and positive, $b_i$ is negative and $|b_i|$ decays as $1/i$, therefore $k_n$ tends to $-\infty$.

This implies that all hitting times are in fact infinite.