I'm studying Markov chains, and I don't understand the solution of the following exercise. It is in Brzezniak, & Zastawniak. "Basic Stochastic Processes."
Excercise 5.26 Show that if $i$ is recurrent and $i\to j$, then $j\to i$.
$i\to j$ means that a state $i$ communicates with a state $j$, i.e. $$P(\xi_n=j \,\operatorname{for some}\,n\geq 0|\xi_0=i)>0,$$ where $(\xi_n)$ is a given Markov chain.
Solution Suppose that $\xi_0=i$. Denote by $\tau_k$ the minimum positive time when the chain enters state $k$, i.e. $$\tau_k=\operatorname{min}\{n\geq 1:\xi_n=k\}.$$ Then, $P\{\tau_j<\tau_i\}=:\epsilon>0$ if $i\to j$ and $i\neq j$. If not $j \to i$, then it would be impossible to return to $i$ with probability at least $\epsilon>0$. But this cannot happen as $i$ is recurrent state. Indeed, $$1=P(\tau_i<\infty)=P(\tau_i<\infty|\tau_j<\tau_i)P(\tau_j<\tau_i)+P(\tau_i<\infty|\tau_j\geq\tau_i)P(\tau_j\geq\tau_i).$$ The second term on the right-hand side is $\leq 1-\epsilon<1$, while the first factor in the first term is equal to $0$ (since not $i\to j$). This is a contradiction.
I think removing "Suppose that $\xi_0=i$." from the first line of the solution, and then adding a conditioning set $\{\xi_0=i\}$ to every probabilities (e.g. write $P(\tau_j<\tau_i|\xi_0=i)$ instead of $P\{\tau_j<\tau_i\}$) would be clearer (I'm not sure it's right to do that, though).
My question is this: Why is $\epsilon>0$?
I've tried to figure it out for several hours, but I couldn't find any clue.