Show $1$ is a persistent null state in $\left(\begin{smallmatrix}1/2&1/2&0\\0&0&1\\\frac{1}{n+1}&0 &\frac{n}{n+1}\end{smallmatrix}\right)$

Question

A three-state time-inhomogeneous Markov chain has the transition matrix: $$\left(\begin{array}{ccc} 1/2 & 1/2 & 0 \\ 0 & 0 & 1 \\ \frac{1}{n+1} &0 &\frac{n}{n+1} & \end{array} \right)$$ where $P(n)$ is the transition matrix at step $n$.

How do I show that state $1$ is a persistent null state? what are the first steps?

Answer 1

For persistence I calculate $f_{ij}(n)$, the probability of returning to state $i$ for the first time, given we start in state $j$ in $n$ steps. This tells us: $$ f_{11}(1) = \frac{1}{2} $$ $$ f_{11}(2) = 0 $$ We move like $1 \to 2 \to 3$ $$ f_{11}(3) = \frac{1}{2}\cdot \frac{1}{3+1}= \frac{1}{8} $$ We now traverse the chain like $1 \to 2 \to 3 \to 3 \to 1$ $$ f_{11}(4) = \frac{1}{2}\cdot \frac{3}{3+1}\frac{1}{4+1} = \frac{12}{30}=\frac{2}{5}. $$ We now traverse the chain like $1 \to 2 \to 3 \to 3 \to 3 \to 1$ $$ f_{11}(5) = \frac{1}{2}\cdot \frac{3}{3+1}\frac{4}{4+1}\frac{1}{5+1} =\frac{3 \cdot 4}{2\cdot 4 \cdot 5 \cdot 6}=\frac{3}{60}=\frac{1}{20}. $$ Continuing in this manner we find for $k>2$ by cancellation of terms in the enumerator and denominator: $$ f_{11}(k) = \frac{1}{2}\cdot \frac{3}{3+1}\frac{4}{4+1} \dots \frac{k-2}{k-1}\frac{k-1}{k}\frac{1}{k+1}= \frac{1}{2} \cdot \frac{3}{k} \cdot \frac{1}{k+1} $$

---

We get by definition of persistence in my book (Grimmet and Stirzaker definition (1) on page 220): $$ f_{11} = \sum_{n=1}^\infty f_{11}(n)$$ If $f_{11}=1$ we know state $1$ is persistent. We will determine the sum by using the values from the previous section: $$ f_{11}= \frac{1}{2} + 0 + \frac{3}{2} \sum_{k=3}^\infty \frac{1}{k} \cdot \frac{1}{k+1}$$ $$=\frac{1}{2} + \frac{3}{2} \sum_{k=1}^\infty \frac{1}{k} \cdot \frac{1}{k+1} - \frac{3}{4} - \frac{1}{4}= \frac{1}{2}+ \frac{3}{2}\cdot 1 - 1=1. $$

Since by telescoping we know: $$S_n= \sum_{k=1}^n \frac{1}{k} \cdot \frac{1}{k+1}=\sum_{k=1}^n \frac{1}{k}- \frac{1}{k+1} =1 - \frac{1}{n+1} $$ And thus $S_n \to 1$ as $n \to \infty$.

---

Now we know that state $1$ is persistent, we can also calculate the mean recurrence time for state $1$ (definition (7) page 222 of Grimmet and Stirzaker: $$ \mu_1 := \sum_{k=1}^\infty k f_{11}(k)$$ This gives us: $$ \mu_1 = 1 \cdot \frac{1}{2} + 2 \cdot 0 + \frac{3}{2}\sum_{k=1}^\infty k \cdot \frac{1}{k (k+1)} > \sum_{k=1}^\infty \frac{1}{k+1}=\infty $$ Which is a divergent minorant. By the comparison test the mean recurrence time is infinite and hence this state is null recurrent/persistent.

Answer 2

Some hints for how to proceed, although I acknowledge that these may be unhelpful since I don't know what your background knowledge is:

Persistence: The limit of $P$ you mentioned in the comments is a bit of an oversimplification; the chain will never actually alternate between states 2 and 3 permanently. Try computing the probability that a chain started at state 3 will always make the choice to visit state 2 at every opportunity. (Or, consider the sum of the probabilities of the chain transitioning from state 3 to state 1 and use a Borel-Cantelli lemma.)

"Null": A path from state 3 to state 1 will always have the same structure: some whole number of interim visits to state 2 first, followed by eventually moving to state 1 (as you proved above). Find the probability of a path of that structure having length $2n + 1$, and use that formulation to compute the expected length of such a path and show that it is $\infty$.