Problem 8.25 in the third edition of Probability and Measure by Billingsley (1995, p. 142) is as follows:
Suppose that an irreducible [Markov] chain of period $t>1$ has a stationary distribution $\{\pi_j\}$. Show that, if $i\in S_{\nu}$ and $j\in S_{\nu+\alpha}$ ($\nu+\alpha$ reduced modulo $t$), then $$\lim_np_{ij}^{(nt+\alpha)}=\pi_j.\tag{$\diamondsuit$}$$ Show that $$\lim_n n^{-1}\sum_{m=1}^np_{ij}^{(m)}=\pi_j/t\quad\text{for all $i$ and $j$.}\tag{$\clubsuit$}$$
Here, the sets $\{S_0,\ldots S_{t-1}\}$ form a partition of the state space with the property that $p_{ij}>0$ only if $i\in S_{\nu}$ and $j\in S_{\nu+1}$ for some $\nu$ ($\nu+1$ reduced modulo $t$). The existence of such a partition has been previously established in Problem 8.24.
My concern is that I think this statement is simply not true. Indeed, assuming that the state space is finite (so that sums and limits commute), the sum of the left-hand side of ($\clubsuit$) is $$\sum_{j}\lim_n n^{-1}\sum_{m=1}^np_{ij}^{(m)}=\lim_nn^{-1}\sum_{m=1}^n \underbrace{\sum_jp_{ij}^{(m)}}_{=1}=1,$$ whereas the sum of the right-hand side is $$\sum_{j}\pi_j/t=1/t<1,$$ given that the $\{\pi_j\}$ are stationary probabilities and $t\geq 2$.
A specific counterexample is the Markov chain given by the following transition matrix: \begin{align*} \begin{array}{c|cc} &a&b\\ \hline a&0&1\\ b&1&0 \end{array} \end{align*} It is easy to see that this Markov chain is irreducible of period $2$ and its unique stationary distribution is $(1/2,1/2)$. Moreover, the state space decomposes as $S_0=\{a\}$ and $S_1=\{b\}$. Yet, all limits of the form ($\diamondsuit$) are $1$ instead of $1/2$.
What’s going on here? I am aware that this otherwise fantastic textbook has minute typos here and there, but this seems to be a major one. The text of this problem is consistent across editions: I checked the second edition—which, surprisingly for an earlier version, seems to contain fewer errors in general—and this problem appears in the same way.
Can anyone confirm whether this is an erroneous claim, indeed, and if so, what should the correct form look like? Any feedback is appreciated.
UPDATE: After some tentative numerical experimentation, I can now conjecture that the correct form should be
- $t\pi_j$ instead of $\pi_j$ on the right-hand side of ($\diamondsuit$);
- $\pi_j$ instead of $\pi_j/t$ in the right-hand side of ($\clubsuit$).
It seems that the author was off only by the scale.
Do you agree?
UPDATE 2: I managed to prove rigorously that the conjecture I had formulated in the first update holds: The expressions for $\pi_j$ in both formulae are off by a factor of $t$.
I also figured out the most plausible reason for this mistake. Consider the partition of the state space $\{S_0,\ldots, S_{t-1}\}$ mentioned above and let $q_{ij}\equiv p_{ij}^{(t)}$ denote the transition probabilities for $t$ units of time. One can show that if the system starts out from $i\in S_{\nu}$ (where $\nu\in\{0,\ldots,t-1\}$), then $p_{ij}^{(kt)}>0$ implies that $j\in S_{\nu}$ for any integer $k$. In words, if the system starts from a particular equivalence class, then it must regularly return to this same equivalence class in $t,2t,3t,\ldots$ periods. This implies that each equivalence class $S_{\nu}$ constitutes a proper subchain under the $t$-fold transition probabilities $\{q_{ij}\}_{i,j\in S_{\nu}}$.
Now, one can show also that the stationary probabilities satisfy $\sum_{j\in S_{\nu}}\pi_j=t^{-1}$ for each $\nu$. Intuitively, the system visits each equivalence class equally often under the stationary distribution (which is not surprising, given that the system cycles through the equivalence classes by their construction). Now, this yields that if $$\widehat{\pi}^{\nu}_j\equiv t\pi_j\quad\forall j\in S_{\nu},$$ then the $\{\widehat{\pi}^{\nu}_j\}_{j\in S_{\nu}}$ constitute a stationary distribution in the subchain $(S_{\nu},\{q_{ij}\}_{i,j\in S_{\nu}})$ for each $\nu$.
The source of the mistake is now clear: Billingsley must have meant the redefined stationary probabilities $\widehat{\pi}_j^{\nu}$ when he wrote $\pi_j$ in ($\diamondsuit$) and ($\clubsuit$).