If $(X_t)$ is a Markov chain then $P(X_t=i|X_0=i)=\sum\limits_{k=1}^tP(X_1 \neq i, X_2\neq i,...,X_k=i,X_t=i|X_0=i)$

117 Views Asked by At

So in my textbook I have the following assertion stated as a theorem $$p_{i,i}^{(t)}=\sum_{k=1}^tf_{i,i}^{(k)}p_{(i,i)}^{(t-k)},$$

where $f_{i,i}$ denotes the first return time to the state $i$.

Then, in the first step, we have, by definition

$$p_{i,i}^{(t)}=\Bbb P(X_t=i|X_0=i),$$

but then this is where the confusion lies. This is equal to

$$\sum_{k=1}^t \Bbb P(X_1 \neq i, X_2\neq i,...,X_k=i,X_t=i|X_0=i).$$

What have they done at this step? The book just cites it as "(law of total probability)" but I have looked around on the Internet trying to see how they have used it but to no avail. I am anticipating the answer to my little problem to be very trivial but my brain just cannot connect the dots so any help here would be appreciated.