Theorem: $P_x(T_y^{k}<\infty) = \rho_{xy}\rho_{yy}^{k-1}$ where $T_y^{k} = \text{inf}\{n>T_y^{k-1}:X_n=y\}$, $T_y^0 = 0$ and $\rho_{xy} = P_x(T_y<\infty)$.
Proof: $k=1$ case is trivial, so suppose $k\geq 2$. Let $Y(w) = 1$ if $w_n = y$ for some $n\geq 1, Y(w)=0$ o.w. If $N = T_y^{k-1},$ then $Y\circ \theta_{N}=1$ if $T_y^k<\infty$. The strong Markov property implies
$$E_x (Y\circ \theta_N|F_N)=E_{X_N}Y\text{ on }\{N<\infty\}$$
On $\{N<\infty\}, X_N = y$, so the RHS above $= P_y(T_y<\infty)=\rho_{yy}$ and thus
$$P_x(T_y^k<\infty) = E_x(Y\circ \theta_N;N<\infty) = E_x(E_x(Y\circ \theta_N|F_n);N<\infty)) = E_x(\rho_{yy};N<\infty) = \rho_{yy}P_x(T_y^{k-1}<\infty)$$
The questions are too simple. Why is $P_x(T_y^k<\infty) = E_x(Y\circ \theta_N;N<\infty)$? By definition, $Y\circ \theta_{N}$ is the indicator of $T_y^k<\infty$, so shouldn't it be $P_x(T_y^k<\infty) = E_x(Y\circ \theta_N)$? Also, at the last step, why is $\rho_{yy}$ a constant so that it comes out of the expectation? Thanks.
Let's show that $$f(\omega)=1_{\{T_y^k<\infty\}}(\omega)=Y\circ \theta_N(\omega)1_{\{N<\infty\}}(\omega)=g(\omega).$$ If $T_y^k(\omega)<\infty$,i.e.$f(\omega)=1$, then $N(\omega)<\infty$ and $Y\circ \theta_N(\omega)=1$ hence $g(\omega)=1$. If $T_y^k(\omega)=\infty$,i.e.$f(\omega)=0$, we need to show that $g(\omega)=0$. Suppose not, i.e, $g(\omega)=1$, then $N(\omega)<\infty$ and $Y\circ \theta_N(\omega)=1$, so $T_y^k(\omega)<\infty$, a contradiction.
Intuitively, $N<\infty$ is needed. Just as the above indicated, on $\{N=\infty\}$, we must have $T_y^k=\infty$ whether $Y\circ \theta_N=1$ or not. From another aspect, we can't well define $\theta_{\infty}$!
$\rho_{yy}=P_y(T_y<\infty)$ is a probability, a number, a constant if we fix $x$ and $y$, so it can be out of the expectation.