Let C be a finite closed set. Then C contains a recurrent state. Clarification about $E_xN(y) = \rho_{xy}/(1-\rho_{yy})$.

Question

I'm not sure how $E_xN(y) = \rho_{xy}/(1-\rho_{yy})$ in the proof below. This is from Durrett's Probability Theory and Examples. I'd appreciate it if anyone could explain it to me. Thanks!

Answer 1

The result is from equation (6.4.1) in the aforementioned textbook, on page 246, right below Exercise 6.4.1. For completeness, I'll recreate the proof here, with extra descriptions.

Denote $T^k_y$ be the time of the $k$th visit to state $y$. denote $N(y) = \sum^\infty_{n=1} 1\{X_n = y\}$ to be the number of visits to $y$ (at positive times). If state $y$ is transient, then

$$ \mathbb E_x N(y) = \sum^\infty_{k=1} P_x [N(y) \ge k] = \sum^\infty_{k=1} P_x(T^k_y < \infty) = \sum^\infty_{k=1} \rho_{xy} \rho_{yy}^{k-1} = \cfrac {\rho_{xy}} {1-\rho_{yy}} < \infty. $$

The first equality follows from the fact that $N(y)$ is nonnegative and discrete by construction and therefore we have a neat expression for computing its expectation by summing over the tail (or complementary) cumulative distribution function.

Here's an intuitive explanation for the second equality... for fixed $k$, if $N(y) \ge k$, then by definition the number of visits to $y$ in positive (finite) time is greater than $k$, which is equivalent to saying the time of the $y$th visit, $T_y^k$, is finite.

An intuitive explanation for the third equality is that, in order to make $k$ visits to $y$, we first have to go from $x$ to $y$ and then return to $y$ exactly $k-1$ times.

And finally, the last part follows since we have a geometric series. By hypothesis, $\rho_{yy} < 1$.