Need name of the mentioned theorem in theory of Markov Chains with its proof.

Question

it this true that for an irreducible and aperiodic finite-state Markov chain with stationary distribution $π,$ the mean return time to each state is finite and equal to the inverse of the stationary probability of that state, i.e., $µ_i = 1/π_i$? I'm looking for a proof. Thank you

Answer 1

we know that $$ \mu_x(y) := \sum_{n = 0}^\infty P_x(X_n = y, \tau_x > n) $$ defines a stationary measure, and we have $$ \begin{aligned} \sum_y \mu_x(y) &= \sum_y \sum_{n = 0}^\infty P_x(X_n = y, \tau_x > n) \\ &= \sum_{n = 0}^\infty\sum_y P_x(X_n = y, \tau_x > n) \\ &= \sum_{n = 0}^\infty P_x(\tau_x > n)\\ &= E_x \tau_x, \end{aligned} $$ where $\tau_x = \inf\{n, X_n = x\}$ is the return time. On the other hand, $\pi(x) := \mu_x(x) / \sum_y \mu_x(y) = \mu_x(x) / E_x \tau_x$ is the unique stationary distribution, and $\mu_x(x) = \sum_{n = 0}^\infty P_x(X_n = x, \tau_x > n) = 1$.