Discrete Time Markov Chain almost sure convergence

68 Views Asked by At

How can I prove that following quantity $$ \frac1m\sum _{i=1}^m \mathbb1\{X(i)=j|X(0)=j \} $$ converges almost surely to $\frac{1}{u_{jj}}$, where $X(n)$ is DTMC, $u_{jj}$ is expected number of steps getting back to state $j$ given $X(0)=j$ and $\mathbb1$ is indicator function.