I'm reading the book Markov chains and Mixing times by Levin and Peres.
I got stuck in one line of the book. Here is it:
I don't understand why we can get (1.21). In this post, people said it came from Markov property but I don't know how we use it.
Understanding the proof of stationary distribution of a markov chain
Any help is highly appreciated.
First note that if $x=z$, then both sides are zero, so we may assume $x \neq z$.
It's just a property of conditional probability that $$\mathbb{P}_z\{X_t=x, X_{t+1}=y, \tau_z^+ \geq t+1\} = \mathbb{P}_z\{X_t=x, \tau_z^+ \geq t+1\}\cdot \mathbb{P}_z\{X_{t+1}=y \mid X_t=x, \tau_z^+ \geq t+1\}$$ so it suffices to show $$\mathbb{P}_z\{X_{t+1}=y \mid X_t=x, \tau_z^+ \geq t+1\} = P(x,y).$$
Now, $$\{\tau_z^+ \geq t+1\} = \{X_1\neq z, X_2 \neq z, \ldots, X_t \neq z\}$$ so that $$\mathbb{P}_z\{X_{t+1}=y \mid X_t=x, \tau_z^+\geq t+1\} = \mathbb{P}_z\{X_{t+1}=y\mid X_t=x, X_1\neq z, X_2\neq z, \ldots, X_{t-1} \neq z\}$$ (note I've used the assumption that $x\neq z$ here)
It's the Markov property that allows us to simplify this to $$\mathbb{P}\{X_{t+1}=y \mid X_t=x\}$$ which by homogeneity is $P(x,y)$.