Let $X$ be a Markov chain with a state space $S={\{0,1,2,... \}}$ and a transition matrix $P$ with given $p_{i,0}=\frac{i}{i+1}$ and $p_{i,i+1}=\frac{1}{i+1}$, for $i=0,1,2,...$. Find out which states are transient, null and not-null. Find stationary distribution.
I have that $p_{0,0}=0, p_{1,0}=\frac{1}{2}, p_{2,0}=\frac{2}{3}, p_{3,0}=\frac{3}{4}$ and $p_{0,1}=1, p_{1,2}=\frac{1}{2}, p_{2,3}=\frac{1}{3}, p_{3,4}=\frac{1}{4}$.
I think that all states are persistent, so no states are transient.
There is a theorem which says that a state is null $\iff \lim_{n \to \infty} p_{ii}(n)=0$, so state $0$ is null. I guess all the other states are non-null but I don't know how to prove it. Also I don't know how to find stationary distribution.
Please help me with the above exercise, correct me where I'm wrong, and send any tips to the rest. Any will be much appreciated.
You need to realise that since $p_{0,1} = 1$ and $p_{i,0} + p_{i,i+1} = 1$ then $p_{0,j}$ must be $0$ for all $j \ne 1$ and $p_{i,k}$ must be $0$ for all $k \ne 0 \mbox{ or } i+1$ .
Let $\pi_j = \frac{e^{-1}}{j!}$ for all $j \ge 0$. Then $\pi_j$ is the stationary distribution because
\begin{eqnarray} \sum_{i=0}^\infty \pi_i &=& 1 \ ,\\ \sum_{i=0}^\infty \pi_i p_{i,j} &=& \frac{1}{j} \pi_{j-1} = \pi_j \ \mbox{ for $j \ge 1$, and}\\ \sum_{i=0}^\infty \pi_i p_{i,0} &=& \sum_{i=0}^\infty \frac{i}{i+1} \pi_i \\ &=& \sum_{i=0}^\infty \left(1 - \frac{1}{(i+1)}\right)\pi_i\\ &=& e^{-1}\sum_{i=0}^\infty \left(\frac{1}{i!} - \frac{1}{(i+1)!}\right)\\ &=& \pi_0\ \ . \end{eqnarray} Consequently, all states are positive recurrent—none are transient or null recurrent.