Show that $W(Y_{\tau \land k})$ is a martingale where Y is a Markov chain and that it converges almost surley

39 Views Asked by At

My questions refers to this paper by T. Lyons. I'm quite new to stochastic processes.

Let $(Y_n)_{n \in \mathbb{N}}$ be a time-homogenous, recurrent, reversible Markov chain with statespace $X$ and transitions $p_{ij}$ for $i,j\in X$.
For reversibility there exist strictly positive weights $\pi_i, i\in X$ with $\pi_ip_{ij}=\pi_jp_{ji}$.
We denote the conditioned expected value and probability on $i\in X$ as $\mathbb{E}^i$ and $\mathbb{P}^i$.

Furthermore we have a function $W:X \to \mathbb{R}$ satisfying:
(i) $W_{i_0}=0$ for a certain $i_0 \in X$
(ii) $W\neq 0$
(iii) $W_i=\sum_{j\in X}p_{ij}W_j$ for $i\neq i_0$
(iv) $\sum_{i,j\in X}\pi_ip_{ij}(W_i-W_j)^2<\infty$

Let $\tau$ be the first hitting time of $i_0$

First question: Why is $W(Y_{\tau \land k})$ a $\mathbb{P}^i$ martingale?

Second question: Why does $\lim_{k \to \infty}W(Y_{\tau \land k})=0$ a.s. hold?


My thoughts:
Regarding the first question my idea was to show that $W(Y)$ is a martingale and than make use of the optional stopping theorem. But somehow I'm confusing myself with this "under $\mathbb{P}^i$" condition. Let $\Sigma_k=\sigma({W(Y_{\tau \land l})}|l=1,...k)$ the induced sigma algebra. Do I have to show that for every $A \in \Sigma_s$ it holds that $$\Bbb{E}^i[\Bbb{E}[W(Y_{\tau \land m})|\Sigma_s]*\Bbb{1}_A]=\Bbb{E}^i[W(Y_{\tau \land s})*\Bbb{1}_A]$$ when $s \leq m$?

Regarding the second question I feel like we do need irreducibility? Why should $\Bbb{P}^i[\tau < \infty]=1$ if there is no way between $i$ and $i_0$? Is this somehow implicitly given?