A relation between first passage time and occupation time

Question

Let's think about a discrete time Markov chain $X_t$ with only one recurrent state. Let $T$ be the random variable that is the number of steps taken from a given state $i$ to the recurrent state (ie. the first passage time to the recurrent state) and $\Phi_k(n) = \sum_{t=0}^n I(X_t = k | X_0 = i)$ be the random variable of occupation time of state $k$ during $n$ steps, where $I$ is the indicator function. Today, in a class on stochastic process, I learned a relation between them : $T=\sum_k \Phi_k(T)$. I'm very confused with this. Is it really legal? How the same random variable $T$ can be appeared in two places in an equation? I have no idea how to interpret the relation. Could someone explain it in plain English?

Answer 1

Yes, this is legal. The right-hand side is a function of a random variable, which again yields a random variable, and the equation states that the random variables on its two sides are equal, which is also a meaningful statement.

The statement isn't quite correct, however; the left-hand side should be $T+1$, or the summation on the right-hand side should be restricted to the non-recurrent states. The process was in exactly one state at any given time, so the sum on the right-hand side contains exactly $T+1$ ones.

Answer 2

This is known as an absorbing Markov chain. Let $k$ be the absorbing state, that is, $\mathbb P(X_{n+1}=k\mid X_n=k)=1$. Then for each state $i\ne k$ we have the hitting time $$T_i := \inf\{n>0:X_n=k\mid X_0=i).$$

The $\Phi_k$ isn't standard notation to my knowledge, but it is true that $$T_i = \sum_{m=1}^{T_i} m\cdot\mathsf 1_{\{T_i=m\mid X_0=i\}},$$ since each term would be zero except for $m=T_i$, which would simply be $T_i$.