Consider a finite state (no. of state $N$) Markov chain $\{X_n\}$ (all the random variables are bounded) such that there is a state $i*$ such that
$$ \sum_{i=1}^{N}p_{ii*}^{(n)} > 0$$ for all $i$.
Then I see a paper claims that
$$\lim_{t\to\infty} \frac{1}{t}E\left[\sum_{k=1}^{t}X_k | X_0 =i\right]$$ is well defined and the value does not depend on $i$. Why ?
We can't use ergodic theorem here directly. In order to independent of the initial state we just need that starting from $i$ the markov chain reaches $j$ with probability $1$ for any $i,j$.