Suppose $P$ is the transition matrix of an infinite Markov chain. Show that if state $i$ is recurrent and it does not communicate with $j$ then $P_{ij}=0$.
I try to use the contradiction by assuming the $P_{ij}>0$. But it does not seem to work.
I get $$P_{ij}^{k+n}\geq P_{ij}^{k}P_{ii}^{n}$$ and $$\sum_{n=1}^{\infty}P_{ij}^{k+n}\geq \sum_{n=1}^{\infty}P_{ij}^{k}P_{ii}^{n}=P_{ij}^{k}\sum_{n=1}^{\infty}P_{ii}^{n}=\infty$$ by the recurrence of $i$.
How to get the $P_{ji}>0$?
It is not necessarily true that $P_{ji}>0$ under your assumptions. Indeed, consider a three state Markov chain with $P_{12}=P_{23}=P_{31}=1$ and $i=1$ and $j=2$. All states are recurrent and $P_{12}>0$ but $P_{21}=0$.
Instead, what needs to be shown is that if $P_{ij}>0$ and $i$ is recurrent, then there exists some sequence of states from $j$ back to $i$ which have positive transition probabilities. But this is clear, since if such a sequence didn't exist then once the Markov chain reached $i$ it could transition to $j$ and never reach $i$ again, contradicting recurrence of $i$.