Prove that $i$ is an accesible state in a markov chain

Question

Let $i$ be a recurrent state of an homogeneous markov chain such that the state $j$ is accesible from $i$ (that is $\exists$ $k\ge 1$ such that $p_{ij}(k)>0$)

Prove that $i$ is accesible from $j$ i.e. $\exists$ $m\ge 1$ such that $p_{ji}(m)>0$

Intuitively I can see why this result holds but I haven´t been able to prove it.

I tried to do it by contradiction: suppose that $i$ is not accesible from $j$ i.e. $\forall m\ge 1$ $p_{ji}(m)=0$ One definition of recurrent state is that $\sum_{n=1}^{\infty}p_{ii}(n)=\infty$

By chapman-kolmogorov $p_{ii}(n)\ge p_{ij}(k)p_{ji}(n-k)$ I don´t know how can I procced with this

I would really appreaciate if you can help me wiht this problem

Answer 1

Since $i$ is recurrent, the probability of ever returning there is 1. Conditioning on the state after $k$ steps of the Markov Chain gives $$1=P(\text{ever return to }x)=\sum_lP(\text{ever from }l\text{ to }x)p_{il}(k).$$ Now, $p_{ij}(k)>0$. So since the $p_{il}(k)$ sum to one, and $P(\text{ever from }j\text{ to }x)$ is smaller or equal to 1, this equality shows that $P(\text{ever from }j\text{ to }x)$ must be one for the equality to hold. Therefore, an $m$ must exist such that $p_{ji}(m)>0$.

Answer 2

If $i$ is not accessible from $j$, then since $p^{k}_{i,j}>0$ for some $k$, there is a nonzero probability that, starting from $i$, we will (end up in state $j$ thus) never return to $i$. This is exactly the definition of a transient state, which is by definition the opposite of a recurrent state.

See https://en.wikipedia.org/wiki/Markov_chain#Transience