Let $\{X_n\} $ be an irreducible Markov chain on a finite state space $S$. Take arbitrary subset $A\subset S$, $\tau_0=\inf\{n:X_n\in A\}$ and $\tau_{n+1}=\inf \{n>\tau_n: X_n\in A\}$. Is $\{\tau_1,\tau_2,\tau_3,...\}$ a Markov chain?
I think that, by letting $Y_n=X_{\tau_n}$, we can use the strong Markov property to show that $\{Y_n\}$ is Markov. So I am wondering if the strong Markov property can also be applied to the sequence of return times as described above. Or maybe that $\{Y_n\}$’s being Markov could imply that return times are Markov as well?
This is not Markov. Take S with 3 elements, {1,2,3}. Take A with 2, {1,2}. Say Xn = n mod 3, so stopping return times are like 1,2,(skip 3), 4, 5, (skip 6) etc. So if you're at a stopping time, you don't know which one you're at. In other words, you don't know if the next stopping time is the next time or you skip one, unless you know your position. Good luck with the rest of your 547 HW!