I am trying to prove the claim: Suppose that for a row stochastic matrix, $P$, there exists an $M < \infty$ such that $m \geq M$ implies $(P^m)_{i,j} > 0$ for all $i,j$. Then $P_{i,j} > 0$ for all $i,j$.
I have attempted the following: \begin{equation*} (P^{m+1})_{i,j} = P(X_{m+1} = s_j | X_0 = s_i) \geq P(X_{m+1} = s_j, X_1 = s_r | X_0 = s_i) \end{equation*} \begin{equation*} = P(X_{m+1} = s_j | X_1 = s_r, X_0 = s_i)P(X_1 = s_r| X_0 = s_i) \end{equation*} \begin{equation*} = P(X_{m+1} = s_j | X_1 = s_r)P_{i,r} = P(X_{m} = s_j | X_0 = s_r)P_{i,r} = P_{i,r}(P^m)_{r,j} \end{equation*} But the inequality in the first line prevents me from creating a meaningful inequality that shows that $P_{i,r} > 0$. I am open to suggestions.
Counterexample: If $P=\pmatrix{0&1\cr 1/2&1/2}$, then $P^2=\pmatrix{1/2&1/2\cr 1/4&3/4}$ and so all higher powers of $P$ also have strictly positive entries.