Proof the statement

Question

Given a finite aperiodic irreducible Markov Chain, prove that for some $n$ all terms of $P^n$ are positive.

I'm little lost in how to prove that, but I know that:

$i)$ If a Markov Chain is irreducible all the states communicate.

$ii)$ If one Markov Chain is aperiodic then all states have period $1$.

$iii)$ If $i\leftrightarrow j\Rightarrow d(i)=d(j)$ where $d$ denotes the period.

$iv)$ If one state $i$ has period $d(i)$, then there exists a integer $N$ such that $\forall n\geq N$ :$P_{ii}^{nd(i)}>0$.

Let $i=0,1,\dots,n$ and $j=0,1,\dots,n$; I think that $i)$ and $iv)$ is enough to show that all diagonal elements are positive, but this does not show that the rest is positive.

I'm stuck

EDIT:

$d(i)=d(j)\Rightarrow i\leftrightarrow j$?

Answer 1

The fact that all states communicate means that for all $i$ and $j$ there is and $n$ such that the $j^{th}$ element of the $i^{th}$ row of $P^n$ is positive. This is because the same element tells the probability that after $n$ steps the chain if let go at state $i$ will get to state $j$.

PS. A state $i$ has period $k$ if any return to state $i$ must occur in multiples of $k$ time steps. Any integer is a multiple of $1$.

Answer 2

1. Note that there is a unique stationary distribution $\pi$ with all components strictly positive, so $P^n_{ij} \to \pi_j > 0$ as $n \to +\infty$ for all $i$, $j$;
2. From the above, we can conclude that for all $i$, $j$, there is some $N_{ij}$ such that for all $n > N_{ij}$, $P^n_{ij} > 0$;
3. What can you say about $\max\limits_{i,j} N_{ij}$?

---

By the way, what you asked in your edit is false. It is trivial to give a counterexample; consider a chain with two absorbing states.