Assume that we have $S_n$ Markov chain with transition matrix $P$. Assume that $S_n \in \{1, \ldots, k\}$ and we have $\tau \in \mathbb{N}$.
We want to estimate probability $\mathbb{P}(S_{n+1} = k \lor \ldots \lor S_{n+\tau} = k | S_n = i)$. It's not hard to get $\mathbb{P}(S_{n+m} = k | S_n = i)$ using matrix. But what about the event there is at least one $m:$ $S_{n+m} = k$? Looks like inclusion-exclusion but in hard manner. Any ideas?
Let $$ T_t=\cases{k&if $\ S_j=k\ \ \text{ for some }\ \ j\ \ \text{ with }\ \ n+1\le j\le t$\\ S_t&otherwise.} $$ Then $\ T_t\ $ is a (time-inhomogeneous) Markov chain for which the state $\ k\ $ becomes absorbing after time $\ t=n\ $, and for which we have the identity $$ \bigvee_{j=1}^m\big\{S_{n+j}=k\big\}=\big\{T_{n+m}=k\big\} $$ For $\ t\ge n+1\ $, the transition matrix $\ \widehat{P}\ $ of $\ T_t\ $ is given by $$ \widehat{P}_{ij}=\cases{P_{ij}&if $\ i\ne k$\\ 1&if $\ i=j=k$\\ 0&if $\ i=k,j\ne k$ .} $$ We then have \begin{align} \mathbb{P}\left(\left.\bigvee_{j=1}^m\big\{S_{n+j}=k\big\}\right|S_n=i\right)&=\mathbb{P}\big(T_m=k\big|S_n=i\big)\\ &=\mathbb{P}\big(T_m=k\big|T_n=i\big)\\ &=\epsilon_i^TP\widehat{P}^{m-n-1}\epsilon_k\ , \end{align} where, for each $\ i\ $, $\ \epsilon_i\ $ is the $\ i^\text{th}\ $ column of the $\ k\times k\ $ identity matrix.