For $i\in\left\{1,2,\ldots,N-1,N,N+1\right\}$ we consider $p_{i},q_{i},r_{i}>0$ such that $p_{i}+q_{i}+r_{i}=1$, then we consider the matrix defined by $$Q=\left[\begin{array}{ccccccccc} 0 &1&0&0&0&\cdots&0&0&0\\ q_{1} &r_{1}&p_{1}&0&0&\cdots&0&0&0\\ 0 &q_{2}&r_{2}&p_{2}&0&\cdots&0&0&0\\ 0 &0&q_{3}&r_{3}&p_{3}&\cdots&0&0&0\\ 0 &0&0&q_{4}&r_{4}&\cdots&0&0&0\\ \vdots &\vdots&\vdots&\vdots&\vdots&\ddots&\vdots&\vdots&\vdots\\ 0 &0&0&0&0&\cdots&r_{N-2}&p_{N-2}&0\\ 0 &0&0&0&0&\cdots&q_{N-1}&r_{N-1}&p_{N-1}\\ 0 &0&0&0&0&\cdots&0&1&0\\ \end{array}\right]$$
Show that there exist $m\in\mathbb{N}$ such that $Q^{m}(i,j)>0$ for all $i,j\in \left\{1,2,\ldots,N-1,N,N+1\right\}$.
To avoid getting in a muddle I renumber all the indices from $0$ to $N$.
I note that the result is not true if $N=1$, so assume $N>1$.
I note that $$Q^m_{i,j}=\sum_{k_1,k_2,\dots,k_{m-1}}q_{i,k_1}q_{k_1,k_2}\dots q_{k_{m-1},j}$$ is a sum of non-negative terms. It is therefore enough to show that at least one of these terms is non-zero.
Now take $m=N$.
I observe that $a_{r,s}\not=0$ precisely when:
$r=0, s=1$
$0<r<N, s=r-1,r,r+1$
$r=N, s=N-1$.
I ask you now to draw a little picture of the graph on the vertices $0,1,\dots N$ with edges exactly where $a_{r,s}\not=0$.
To get a non-zero product I need, for a given $i,j$, to construct a path from $i$ to $j$ of length $N$. As the greatest $|i-j|$ is $N$, and as it is possible to "hover" on any $r\not=0,N$, the existence of such a path is, well, "obvious".