Let $(X_n)$ be a Markov chain with state space E. Show: If a state $i$ is aperiodic then there exists a $n_0\in\mathbb{N}$ so that $p_{ii}^{(n)}>0~\forall~n\geqslant n_0$.
I know that $i\in E$ aperiodic means, that $p(i)=gcd\left\{n>0: p_{ii}^{(n)}>0\right\}=1$.
Could you please show me how I can prove the above statement?
Once again, this is textbook stuff:
Let $D(i)=\{n\geqslant1\mid p_{ii}^{(n)}\ne0\}$, then, by hypothesis, the greatest common divisor of $D(i)$ is $1$. By Bezout identity, there exists finitely many integers $(n_k)_{1\leqslant k\leqslant N}$ in $D(i)$ and some integers $(a_k)_{1\leqslant k\leqslant N}$ such that $$a_1n_1+\cdots+a_Nn_N=1.$$ Assume without loss of generality that $a_k$ is positive for every $k\leqslant M$ and negative otherwise, then the positive integer $$K=|a_{M+1}|n_{M+1}+\cdots+|a_N|n_N,$$ is such that $$K+1=a_1n_1+\cdots+a_Mn_M.$$ Now, the set $D(i)$ is stable by sums and $D(i)$ contains every $n_k$ hence $D(i)$ contains $K$ and $K+1$.
By Euclidean division, each integer $n\geqslant K(K-1)$ can be written as $n=qK+r$ for some nonnegative integers $(q,r)$ such that $q\geqslant K-1\geqslant r$ hence $$n=qK+r(K+1-K)=(q-r)K+r(K+1).$$ Since $q-r$ and $r$ are nonnegative, $K$ and $K+1$ are in $D(i)$, and $D(i)$ is stable by sums, this proves that $D(i)$ contains $n$, thus, $D(i)\supseteq\mathbb N\cap[K(K-1),+\infty)$.