I am given this Markov chain with specific transitions: $$P(0,k)=pq^{k-1}$$ $$P(k,k-1)=1$$ $$p+q=1$$ and I am asked asked to
- prove that all states are positively recurrent, by showing that for any k and l, $E_k(T_l)<\infty$
- find a probability distribution $\mu$ on $N_0$ such that $E_\mu(T_k)=\infty$ for any k
I am specifically asked not to use the ergodic theorem or the stationary distribution....
- First I am wondering why it isn't sufficient to show for (1) that $E_k(T_k)<\infty$ for any k....
- Second, I started to express $E_k(T_l)$ but I do get stuck at some point or at least unsure of how to check I am doing it right:
a) if k>l then $E_k(T_l) = k-l$;
b) assuming k<l
$$E_k(T_l) = \text{time to go from k to 0 + time to go from 0 to l} = k + E_0(T_l)$$ $$E_0(T_l) = \text{time to go from 0 to m + time to go from m to l} = 1 + \sum_{m=1}^{\infty} pq^{m-1} * E_m(T_l) $$ $$= 1 + \sum_{m=1}^{l-1} (m + E_0(T_l)) * pq^{m-1} + \sum_{m=l+1}^{\infty}(m-l)*pq^{m-1}$$ $$= 1 + \sum_{m=1}^{l-1} m * pq^{m-1}+ \sum_{m=1}^{l-1} E_0(T_l) * pq^{m-1} + \sum_{u=1}^{\infty}u*pq^{u+l-1}$$ So $$E_0(T_l) = \frac{1+ \frac{q^l}{p} + \sum_{m=1}^{l-1} m * pq^{m-1}}{1+\sum_{m=1}^{l-1} pq^{m-1}}$$
I would appreciate some help finishing the proof or identifying went it went wrong... Also some help with the second question about the $\mu$.