I have a question about the Markov Chain. We were doing derivation of the Chapman-Kolmogorov Equations, the $n+m$ step state transition probability (please see below):
where $P_{i,j}$ is the probability transition from state $i$ to state $j$. In particular, I am confuse about the part highlighted in yellow.
$P(X_{n+m}=j|X_{n}=k,X_{0}=i)=P^m_{j,k}$
I do understand the time stationary of Markov Chain. What I don't understand is the Memoryless-Property.
So is it true that $P(X_{n+m}=j|X_{n}=k,X_{0}=i)=P(X_{n+m}=j|X_{n}=k)$? The reason is because that $X_0$ is the initial state so that we can ignore it?
PS: What happen if we have another state $X_{n+1}$, and I have $P(X_{n+m}=j|X_{n+1}=b,X_{n}=k,X_{0}=i)$, can I say it is equal to $P(X_{n+m}=j|X_{n+1}=b)$?
Conditioned on any state, the progression of a Markov chain is independent of any earlier state, so yes,
$$ P(X_{n+m} = j \mid X_n = k, X_0 = i) = P(X_{n+m} = j \mid X_n = k) $$
and that means also that you are right, that
$$ P(X_{n+m} = j \mid X_{n+1} = b, X_n = k, X_0 = i) = P(X_{n+m} = j \mid X_{n+1} = b) $$