Let $(X_n)_{n\in\mathbb{N}_0}$ be an irreducible Markov chain with state space $E$ and Transition Matrix $P=(p_{ij})_{i,j\in E}$. Set $$ p_{ii}^{(n)}=\mathbb{P}(X_n=i|X_0=i)=:\mathbb{P}_i(X_n=i). $$ Let $i,j\in E$ with $i\neq j$. By irreducibility there exist $k,\ell\in\mathbb{N}$ such that $p_{ij}^{(k)}>0$ and $p_{ji}^{(\ell)}>0$. Show that $$ p_{ii}^{(k+l)}\geqslant p_{ij}^{(k)}\cdot p_{ji}^{(\ell)}>0. $$
My idea is to use the Chapman-Kolmogorov Equation which is as follows:
Let $(X_n)_{n\in\mathbb{N}_0}$ be a Markov chain with state space $E$ and Transition Matrix $P$. Let $k<m<n$. Then for all $i,j\in E$ it is $$ \mathbb{P}(X_n=j|X_k=i)=\sum_{y\in E}\mathbb{P}(X_n=j|X_m=y)\cdot\mathbb{P}(X_m=y|X_k=i). $$
Now to my proof:
Without loss of generality let $k<\ell$. Using the Markov property and the Chapman-Kolmogorov equation, it is
\begin{align} p_{ii}^{(k+\ell)}&=\mathbb{P}_i(X_{k+\ell}=i)\\ &=\sum_{a\in E}\sum_{b\in E}\mathbb{P}_i(X_{k+\ell}=i,X_k=a,X_{\ell}=b)\\ &=\sum_{a\in E}\sum_{b\in E}\mathbb{P}_i(X_{k+\ell}=i|X_k=a,X_{\ell}=b)\cdot\mathbb{P}_i(X_k=a,X_{\ell}=b)\\ &=\sum_{a\in E}\sum_{b\in E}\mathbb{P}_i(X_{k+\ell}=i|X_{\ell}=b)\cdot \mathbb{P}_i(X_{\ell}=b|X_k=a)\cdot\mathbb{P}_i(X_k=a)\\ &=\sum_{a\in E}\mathbb{P}_i(X_{k+\ell}=i|X_k=a)\cdot\mathbb{P}_i(X_k=a)\\ &=\sum_{a\in E}p_{ia}^{(k)}\cdot p_{ai}^{(\ell)}\\ &\geqslant p_{ij}^{(k)}\cdot p_{ji}^{(\ell)}. \end{align}
It would be great if you could tell me if my proof is ok.
Thank you and with greetings
Yes, your proof is correct.
Alternative argumentation: For
$$P^n := \underbrace{P \cdots P}_{\text{$n$ times}} = (p_{ij}^{(n)})_{i,j \in E}, \qquad n \in \mathbb{N},$$
we have $P^{k+l} = P^k \cdot P^l$. Therefore,
$$p^{(k+l)}_{ii} = \sum_{j \in E} p_{ij}^{(k)} p_{ji}^{(l)} \geq p_{i j_0}^{(k)} p_{j_0 i}^{(l)}$$
for any fixed $j_0 \in E$.