- A Markov process on $E = {1, 2}$ is constructed according to holding time parameters $λ_1 = 2$ and $λ_2 = 4$; the defining Markov chain has transition probabilities $p_{11} = p_{12} = 0.5$ and $p_{21} > = 1$.
(a) Calculate the generator matrix for this process.
(b) What is the variance of the time (starting from state 1) until the process jumps to state 2? Justify your answer.
I've calculated my generator matrix: \begin{bmatrix} -1 & 1 \\ 4 & -4 \\ \end{bmatrix} However I'm a little unsure for part (b). Here is my attempt:
I think the variance is given by $\frac{1}{λ_1^2} = \frac{1}{4}$.
This is because the memoryless property of the exponential holding times mean the remaining time in a state $i$ is distributed exponentially with parameter $λ_i$ and the variance of an exponential distribution with parameter $λ$ is $\frac{1}{λ^2}$.
Is this satisfactory/correct?
The process stays in state $1$ an exponential amount of time with mean $1/\lambda_1$. The process then transitions to state $1$ with probability $1/2$ or to state $2$ with probability $1/2$.
Let $T$ be the time spent in state $1$ until the process reaches state $2$. As a hint, let us try to calculate $\mathbb{E}[T]$. By conditioning on the number of times $N$ the process visits state $1$ before visiting state $2$ we get
\begin{align} \mathbb{E}[T] &= \sum_{n = 1}^\infty \mathbb{E}[T \mid N = n] \mathbb{P}(N = n) \\ &= \sum_{n = 1}^\infty \mathbb{E}[T \mid N = n] (1/2)^n \\ &= \sum_{n = 1}^\infty \mathbb{E}[\text{Erlang-$n$ with parameter $\lambda_1$}] (1/2)^n \\ &= \sum_{n = 1}^\infty \frac{n}{\lambda_1} (1/2)^n = \frac{2}{\lambda_1}. \end{align}
The important steps are the conditioning on the number of times we visit state $1$ before visiting state $1$ and recognizing the third equality. Keeping this hint in mind, can you find $\mathbb{E}[T^2]$ so that you can get $\operatorname{Var}(T) = \mathbb{E}[T^2] - \mathbb{E}[T]^2$?