Given a Markov chain with state space $\Omega$ and transition matrix $P$, and $A\subset \Omega$, define function$f(x)=\Bbb E _x(\tau_A)$, where $\tau_A$ is the "stopping time with respect to a set", meaning the first time the Markov chain reaches a state in $A$.
Obviously, $f(x)=0$ if $x\in A$.
The question is to show $f\left( x \right) = 1 + \mathop \sum \limits_{y \in {\rm{\Omega }}} P\left( {x,y} \right)f\left( y \right)$ for $x\notin A$.
The following is my attempt, however I fell very unsafe about the part circled red. I figured out it has to be this way, but I don't have a clear understanding why the circled part holds rigorously (intuitively it holds). Anyone can help with my confusion? Thank you!
