Let $\{X_n\}$ be a homogenous Markov chain, taking values in N.
$T_i:=\inf\{k\ge0:X_k=i\}$ is the first time when the chain arrives at i. I know that if X is irreducible positive recurrent, then $E_iT_i$ is finite.
Now I want to know some results about $E_j T_i$, the expected time of first arrival at state $i$ starting from state $j$. I think that it should be finite almost surely.
Is there any explicit form of $E_j T_i$, or inequality? And what about an arbitrary initial distribution? Any help would be appreciated.
For a fixed $i$, let $\psi(j) = \mathbb{E}_j T_i$, i.e. the expected time to reach state $i$ starting from state $j$. Then for $\psi(j)$ the following holds:
$$ \psi(j) = \begin{cases} 0\text{,} & j = i\\ 1 + \sum\limits_{k \neq j} p_{j k}\psi(k)\text{,} & j \neq i \end{cases} $$
where $p_{j k}$ denotes the one-step transition probability from $j$ to $k$.
Proof sketch Start from state $j$. If $j = i$, then obviously $T_i = 0$, and so $\mathbb{E}_j[T_i \mid j = i] = 0$. If, however, $j \neq i$, then $T_i = 1 + T^{\prime}_i$, where $T^{\prime}_i$ denotes the remaining time to reach $i$. Invoking the Markov property conditional on the first step, and the chain's homogeneity, we can get the recurrence stated above.
You can generalize the above recurrence using an arbitrary award function depending on which state you end up in. Here, for example, the award function gives $1$ (you can count this $1$ as a penalty though) whenever $j \neq i$, and $0$ when $j = 0$.
Another way you can calculate this is by marking the state you want to reach as an absorbing one, and use the fundamental matrix to calculate the expected time until absorption. There is also a mean first passage matrix. See Grinstead & Snell, pp. 461 onward.