I am reading the following book:
Markov Chains and Invariant Probabilities, by Herndadez-Lerma and Lasserre (p.23)
- Given measure space $(X,\mathcal{B})$
For each $x\in X$ and $B\in \mathcal{B}$, let $$P(x,B)=\mathcal{P}(\xi_{n+1}\in B \mid \xi_n = x).$$ This defines a stochastic kernel on $X$, which means
- $P(x,\cdot)$ is a probability measure on $\mathcal{B}$ for each fixed $x\in X$.
- $P(\cdot,B)$ is a measurable function on $X$ for each fixed $B\in \mathcal{B}$.
The $n-$step stochastic kernel $P^n(x,B)$ can be defined recursively by $$P^n(x,B) = \int_XP(x,dy)P^{n-1}(y,B) = \int_XP^{n-1}(x,dy)P(y,B)$$
with $P^0(x,\cdot) = \delta_x(\cdot)$.
Based on this definition,
$$P^1(x,B) = \int_X P^0(x,dy)P(y,B) = \int_x P(y,B)\delta_x(dy) = P(x,B).$$
$$P^2(x,B) = \int_X P^1(x,dy)P(y,B) = \int_X P(y,B) \int_X P(y,dy)\delta_x(dy)$$
and then we have
$$P^2(x,B) = = \int_X P(y,B) P(y,dy)\delta_x(dy) = P(x,B) \int_X P(y,dy)\delta_x(dy)$$
I have no idea how to say $\int_X P(y,dy)\delta_x(dy) =P(x,B) $.
Moreover, can anyone let me know the meaning of such definition of $P^n(x,B)$ from the perspective of Markov Chain (this definition seems difficult to understand for me)?
$\int f(y) d\delta_x(y)=f(x)$ for any non-negative measurable function $f$.
$P^{n}(x,B)$ is the probability of starting at $x$ at time $0$ and landing inside $B$ in $n$ steps. You have made a mistake with $P^{2}(x,B)$. You should write $P^{2}(x,B)=\int_X P^{1}(y,B) P^{1}(x,dy)$. It is not possible to pull out $P(x,B)$ from this.