Example of Markov chain

Question

I am struggling to solve this problem. I have no idea about how to tackle it. Please pardon me for not showing what I can I do. Thank you.

Let $X= \{X_n; n= 0, 1 \ldots\}$ be a Markov chain with state space $J$ and transition matrix $P.$ Fix a state $i$ and suppose $p(i,i)>0$. Let $$T=\text{inf} \{n\geq 1; X_n \neq i \}$$ Assume that the Markov chain starts in state $i$.

For $j \neq i$ and $n = 1,2, \ldots,$ find $$P_i\{X_T = j, T= n\}$$ and for $j \neq i$ find $$P_i \{X_T = j \}$$

Answer 1

Conditioned on $\{X_0=i\}$, the event $\{X_T=j,T=n\}$ corresponds to $n-1$ transitions from $i$ to $i$ followed by one transition from $i$ to $j$. By the law of total probability, we have \begin{align} \mathbb P(X_T=j\mid X_0=i) &= \mathbb P\left(\bigcup_{n=1}^\infty \{X_T=j,T=n\mid X_0=i\}\right)\\ &= \sum_{n=1}^\infty \mathbb P(X_T=j,T=n\mid X_0=i). \end{align} Can you finish it from there?