The question I get asked is the following, I'm completely stuck on the problem:
Let $i$ be an aperiodic state of a Markov Chain. Show that there exists $N\geq 1$ such that $p_{i,i}(n)>0$ for all $n\geq N$.
Where $p_{i,i}(n)$ denotes the $n$-step transition probability of a $i\to i$ for some $i$ in the state space $S$.
I assume, the state $i$ is also permanent, else it wouldn't work.
$i$ is permanent, so it lies at least on one cycle. Let the cycle lengths be $l_1, l_2, l_3...l_n$. The state $i$ is aperiodic, so $gcd(l_1,l_2,l_3...l_n)=1$. Because of Bezout's identity, there exist integral coefficinets $c_1,c_2,c_3...c_n$ such that $c_1l_1+c_2l_2+c_3l_3+...+c_nl_n=1$. Now let $N=min(l_k)\cdot\sum_{k=1}^{n}(|c_k|l_k)$. State $i$ is reachable in $N$ steps because it is in a form of sum of lengths of used cycles. State $i$ is reachable in $N+m$ steps where $0 < m < min(l_k)$ because $N+m$ can be expressed as a sum of $M$ and $m\cdot(c_1l_1+c_2l_2+c_3l_3+...+c_nl_n)$, which is always positive. Any higher number can be expressed as $N+a\cdot min(l_k)+m$, which is the sum of $N+m$ and $a\cdot min(l_k)$.
Q.E.D.