Expected exit time from state i

Question

In a Markov chain, if I want to find mean exit time, is it better to think about it as the opposite of mean recurrence time or related to expected first passage?

Answer 1

Let $p_{ij}=\mathbb P(X_{n+1}=j\mid X_n=i)$ be the transition probabilities, and define $$\tau = \inf\{n>0: X_n\ne i\mid X_0=i\}.$$ Then for each positive integer $k$, \begin{align} \mathbb P(\tau=k) &= \mathbb P\left(\{X_k\ne i\}\bigcap\cap_{j=1}^{k-1}\{X_j=i\}\mid X_0=i \right)\\ &= \mathbb P(X_k\ne i\mid X_{k-1}=i )\prod_{j=1}^{k-1} \mathbb P(X_j=i\mid X_{j-1}=i)\\ &= (1-p_{ii})p_{ii}^{k-1}, \end{align} so that $\tau\sim\mathrm{Geo}(1-p_{ii})$. It follows that \begin{align} \mathbb E[\tau] &= \sum_{k=1}^\infty k\mathbb P(\tau=k)\\ &= \sum_{k=1}^\infty k(1-p_{ii})p_{ii}^{k-1}\\ &= (1-p_{ii})\sum_{k=0}^\infty (k+1)p_{ii}^k\\ &= \frac1{1-p_{ii}}. \end{align}