Let $(X_t)$ be a continuous Markov chain and $\tau$ a stopping time. How to compute $\mathbb E [e^{-r\tau} \phi (X_\tau)]$?

Question

Let $(X_t)$ be a continuous-time Markov chain on probability space $(\Omega, \mathcal F, \mathbb P)$ such that

• $X_0 = a$ almost surely.

• The state space $V$ is finite and endowed with discrete topology.

• The infintesimal generator is $L: V^2 \to \mathbb R$.

Moreover, we have $\phi: V \to \mathbb R_+$ and $D \subseteq V$. Consider the stopping time $$\tau = \inf \{ t \ge 0 \mid X_t \in D\}$$ Then I'm interested in computing $$\alpha = \mathbb E [e^{-r\tau} \phi (X_\tau)] \tag{1}$$

---

My attempt:

Let $Y = e^{-r\tau} \phi (X_\tau)$. Then $Y$ is a random variable defined by $$\forall \omega \in \Omega:Y (\omega) = e^{-r\tau(\omega)} \phi (X_{\tau(\omega)}(\omega))\tag{2}$$ It follows that $$\alpha = \int Y (\omega) \, \mathrm{d} \mathbb P (\omega) \tag{3}$$

---

In class, I'm given a random variable $Z$ with p.d.f $f$. Then I compute $\mathbb E[Z]$ by $$\mathbb E[Z] = \int zf(z) \, \mathrm{d}z \tag{4}$$, which is usually the Riemann integral.

This is the first time I'm required to compute the integral $(1)$. Could you please elaborate on how to compute $\alpha$ in $(1)$? Any reference is appreciated.

Answer 1

I convert @gt6989b's comment as answer to close this question.

---

How is this different from the Law of the Unconscious Statistician giving $$\mathbb{E}[e^{-r\tau}\phi(X_\tau)] = \int_\mathbb{R} e^{-rt} \phi(X_t) f_\tau(t) dt,$$ where $f_\tau(\cdot)$ is the pdf of the stopping time $\tau$?