Suppose you have a finite-state continuous-time inhomogeneous Markov chain with transition rate $Q(t)$. Further, let us suppose that $Q(t)$ is a piecewise continuous function of $t$. Two questions:
- Let $H_t(i)$ be the time until the chain reaches state $1$ starting from state $i$ at time $t$. How do I argue that $H_t(i)$ is a random variable for any $t,i$?
Presumably, when one deals with these chains, there is a sigma algebra relative to which everything is measurable (or perhaps an increasing sequence of sigma-algebras?) and one needs to argue that $H_t(i)$ is measurable with respect to it.
- How do I argue that $E[H_t(i)]$ is a Borel measurable function of $t$?
Some background: my background in probability theory is not very strong, and yet now I am writing up a report which deals in part with continuous time Markov chains. I only need to do some very basic algebraic manipulations though, but they seem to require 1. and 2. above to be justified.
Here's an answer to the first question: assuming the Markov process $X_s$ is right-continuous, $H_t(i)$ will be a random variable, because we can write $H_t(i)$ as a countable intersection of measurable sets: $$\{H_t(i) < s\} = \bigcap_{\underset{\Large t \leq q< s}{q\in\Bbb Q}} \{X_q=1\}$$