I'm working through a Markov Chain exercise related to mean recurrence times and I was hoping to check my work here. The states of the chain are distributed in an infinite way that I haven't dealt with yet, and while my results end up making intuitive sense to me, I want to be sure my work and logic is correct.
1.) Consider a homogeneous Markov Chain on the state space $S = \Bbb{N}$ given by
$P(X_1 = i \mid X_0 = i) = r, i\ge 2$
$P(X_1 = i-1 \mid X_0 = i) = 1-r, i\ge 2$
$P(X_1 = j \mid X_0 = 1) = \frac{1}{2^j}, j\ge 1$
Classify the states of the chain and find their mean recurrence times.
Answer:
So, first thing I did was to construct the transition matrix so I could better visualize:
$$ \begin{bmatrix} \frac{1}{2} & \frac{1}{4} & \frac{1}{8} &\cdots & \frac{1}{2^j} \\ 1-r & r & 0 & \cdots & 0 \\ 0 & 1-r & r & \cdots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 1-r & \cdots & r \\ \end{bmatrix} $$
Now, here's where I started thinking, since we are considering a stochastic matrix, I know that the summation of each state must = 1. Thus, the only way to have row 1 sum to 1 is to bring j to infinity, so that $\sum\frac{1}{2^j}$ converges to 1.
So, then I started considering the stationary distributions:
$$ \begin{bmatrix}\pi_1 & \pi_2 & \pi_3 & \cdots & \pi_j \\\end{bmatrix} \begin{bmatrix} \frac{1}{2} & \frac{1}{4} & \frac{1}{8} &\cdots & \frac{1}{2^j} \\ 1-r & r & 0 & \cdots & 0 \\ 0 & 1-r & r & \cdots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 1-r & \cdots & r \\ \end{bmatrix} $$
(Of course, $\pi_3$ could also be thought as $\pi_{j-1}$)
Thus, as I work through these (I'll save the time of writing all the work down here), I get stationary distributions of zero, and therefore mean recurrence times of $\infty$. Now, this makes sense intuitively from the need to bring j to infinity in order to create a proper stochastic matrix. If you simply think of the size of the chain and the behavior of $\frac{1}{2^j}$, infinite MEAN recurrence times seem to make sense.
Is this correct? And is it due to that fact that the states are null-recurrent?
I really appreciate any insight!