A discrete time Markov chain defined on the state space $\{0,1,2,\ldots\}$ has transition probabilities $$p_{i,i+1}=p$$ $$p_{i,i}=1-p-q$$ $$p_{i,i-1}=q, \quad i\geqslant1$$ where $$p_{0,0}=1-p$$ $$p_{0,1}=p$$
$$0 <p,q < 1$$ $$p + q < 1$$
Find, for all pairs of values $(p,q)$, the expected time until the first return to state $0$ given that $X_n = 0$.
I'm new to Markov chains, so can anyone please explain in some detail how to obtain the answer to this problem? The preceding problem asked to find the values of $p$ and $q$ for which the stationary distribution exists. For this problem, I found it was necessary that $p$ and $q$ were both less than $\frac{1}{2}$ (from the convergence condition for a geometric series). Does this in any way help me to solve the highlighted problem above?
Let $\nu(0)=1$ and $$ \nu(i) = \left(\frac pq\right)^i,\; i\geqslant1. $$ Then $\nu(0)P_{00} + \nu(1)P_{10}= 1-p + q\cdot \frac pq = q=\nu(0)$ and for $i\geqslant 1$, \begin{align} \nu(i-1)P_{i-1,i} + \nu(i)P_{i,i} + \nu(i+1)P_{i+1,i} &= \left(\frac pq\right)^{i-1}p + \left(\frac pq\right)^i(1-p-q)+ \left(\frac pq\right)^{i+1}q\\ &= p^i\left(q^{-(i-1)} + q^{-i}(1-p-q)+pq^{-i}\right)\\ &= \frac{p^i}{q^{i-1}}\left(1 + q^{-1}-pq^{-1} - 1 + pq^{-1} \right)\\ &= \left(\frac pq\right)^i\\ &= \nu(i), \end{align} so that $\nu$ is an invariant measure for $P$. Assuming $p<q$, the sum $\sum_{i=0}^\infty \nu(i):=C$ is finite. Let $\pi(i) = \frac 1C\nu$, then $\pi$ is a stationary distribution for $P$. We compute $$ \sum_{i=0}^\infty \nu(i) = \sum_{i=0}^\infty \left(\frac pq\right)^i = \frac q{q-p}$$, and hence
$$ \pi(i) = \left(1-\frac pq\right)\left(\frac pq\right)^i. $$
The expected time until the first return to state $0$ is given by $$ \frac1{\pi(0)} = \frac q{q-p}. $$