Suppose $X_n$ is an irreducible aperiodic finite state MC, with $P$ being the transition matrix. Then we know that $P^n$ has all positive entries for some $n\in\mathbb N$. If the state space $S$ of this MC has cardinality $d$, then show that $P^{d^2}$ has all entries strictly positive.
I have seen that $P^{(d-1)^2+1}$ has all entries positive. I do not want to use this.
Is there any way to show that $P^{d^2}$ has all entries positive? I have been told this is trivial. Unfortunately, I cannot figure it out.
I heard the following is true, although this claim has been made without giving a proof:
Take any $i$ in the state space. Then there exists $k\leq d$ so that $p_{ii}^{(k)}>0$. Therefore, $p_{ii}^{(nk)}>0$.
Therefore, $P^{nk}$ has $i$-th column comprising entirely of strictly positive entries.
This last sentence is what I do not get. I took any $j\neq i$ and $p_{ji}^{(nk)}\geq p_{ji}^{(l)}p_{ii}^{(nk-l)}$ for $l=\min\{n:p_{ji}^{(n)}>0\}$. Can I guarantee that $p_{ii}^{(nk-l)}>0$? Will a different choice of $l$ work?
I am afraid I am not exploiting the aperiodicity condition, at all.
Also, if this statement Therefore, $P^{nk}$ has $i$-th column comprising entirely of strictly positive entries. is verified, then basically we have got the answer to our question.