Let
- $(\Omega,\mathcal A)$ be a measurable space
- $I$ be an at most countable set
- $\mathbb F=(\mathcal F_t)_{t\in I}$ be a filtration in $(\Omega,\mathcal A)$
- $X=(X_t)_{t\in I}$ be an $\mathbb F$-adapted stochastic process with values in a measurable space $(E,\mathcal E)$
- $\tau$ be a $\mathbb F$-stopping time
- $X_\tau(\omega):=X_{\tau(\omega)}(\omega)$ for $\omega\in\Omega$
- $\mathcal F_\tau:=\left\{A\in\mathcal A:A\cap\left\{\tau\le t\right\}\in\mathcal F_t\;\text{for all }t\in I\right\}$
I'm curious whether or not we need to force $\tau$ to be finite, if we want to show, that $X_\tau$ is $\mathcal F_\tau$-measurable.
Let's take a look at a possible proof. I will use the following elementary result:
Lemma 1$\;\;\;$Since $I$ is almost countable and $\tau$ is a $\mathbb F$-stopping time, $$\left\{\tau=t\right\}\in\mathcal F_t\;\;\;\text{for all }t\in I\;.$$
Now, let $A\in\mathcal E$ and $t\in I$. If $s\in I$ with $s\le t$, then $$\left\{\tau=s\right\}\in\mathcal F_t$$ by (Lemma 1) and, since $X$ is $\mathbb F$-adapted, $$X_s^{-1}(A)\in\mathcal F_t\;,$$ i.e. $$X_s^{-1}\cap\left\{\tau=s\right\}\in\mathcal F_t\;.$$
Since $I$ is almost countable, $$X_\tau^{-1}(A)\cap\left\{\tau\le t\right\}=\bigcup_{s\in I:s\le t}\left(X_s^{-1}\left(A\right)\cap\left\{\tau=s\right\}\right)\in\mathcal F_t$$
Is there any part of the proof which doesn't work until $\tau<\infty$ is enforced?
$X_{\tau}(\omega)$ is not well-defined for $\omega \in \{\tau=\infty\}$. Since the limit
$$X_{\infty}(\omega) := \lim_{i} X_i(\omega)$$
does, in general, not exist, we cannot simply set $X_{\tau}(\omega) := X_{\infty}(\omega)$ for $\omega \in \{\tau=\infty\}$. One possibility to fix this is defining
$$X_{\tau}(\omega) = 0$$
for $\omega \in \{\tau=\infty\}$, i.e.
$$X_{\tau}(\omega) := X_{\tau(\omega)}(\omega) 1_{\{\tau<\infty\}}(\omega).$$
Then, using your argumentation, it is not difficult to see that this random variable is $\mathcal{F}_{\tau}$ measurable.