The two notations highlighted above are not defined before. I have browse through all pages before to ensure this. I tried to find out their definitions in the context but failed.
Do these notations too common to be referred to especially? But I have never encountered them at least in Kai Lai Chung’s A Course in Probability Theory.
Thank you for any answers or comments! I have been stuck for a long time.
ps. These notations also appear in the 4th edition of the book, but are not defined not.
The standard interpretation of the notation $P_x$ and $\mathbb E_x$ in the context of Markov chains is that they are probabilities and expected values under the assumption that the starting state is $x$. This seems to be consistent with how they're used here. (Note that $P_x$ and $\mathbb E_x$ should be interpreted as special cases of $P_\mu$ and $\mathbb E_\mu$.)
So, for example, in $$P_x(X_{m+n}=z) = \sum_y P_x(X_m = y) P_y(X_n = z)$$ we interpret $P_x(X_{m+n}=z)$ as the probability that $X_{m+n}=z$ when the Markov chain is initialized at $X_0 = x$. (I wouldn't want to call this a conditional probability $P(X_{m+n}=z \mid X_0 = x)$ only because talking about the probability of $X_0 = x$ might not make sense without saying how the Markov chain is initialized, but in spirit it is a lot like that conditional probability.)
In this equation, we can think of $P_x(X_{m+n}=z)$ as the probability of going from $x$ to $z$ in $m+n$ steps. On the right, $P_x(X_m=y) P_y(X_n = z)$ is the probability of going from $x$ to $y$ in $m$ steps times the probability of going from $y$ to $z$ in $n$ steps, so it is the probability of going from $x$ to $z$ in $m+n$ steps such that the $n^{\text{th}}$ step goes through $y$. By summing over all $y$, we get the left-hand side.