Supermartingale that converges almost surely to a constant

41 Views Asked by At

Question: Let $X$ be a Markov chain with countable state space $S$ and a transition matrix $P$. Suppose that $X$ is irreducible and persistent and that $\phi: S \to S$ is bounded function satisfying $$ \sum_{j \in S}p_{i,j}\phi(j) \le \phi(i) $$ for $i \in S$. Show that $\phi$ is a constant function.

My Attempt (so far): First, we have that $\phi(X_n)$ is a supermartingale since $E[\phi(X_{n+1})|\mathcal{F}_n] = \sum_{j \in S}p_{X_n,j} \phi(j) \le \phi(X_n)$. By the boundedness of $\phi$ this supermartingale converges almost surely. That is, $\lim_{n \to \infty}\phi(X_n)$ exists.

Also, since this chain is irreducible we know that the event $\{X_n = i \} \; i.o$ has probability one. However, now I'm not completely sure how to proceed. If anyone has any ideas - I would greatly appreciate it. Thanks.