Let $X$ be a Markov chain with state space $\mathcal{S}$ and denote $\mathbb{N} := \{ 0, 1, \cdots\}$. We know that for any stopping time $\tau < \infty$ and any bounded measurable function $\phi : \mathcal{S} \longrightarrow \mathbb{R}$, there exists a function $\psi : \mathbb{N}\times \mathcal{S} \longrightarrow \mathbb{R}$ such that
\begin{align} \mathbb{E}[\phi(X_{\tau+1}) \; | \; \mathcal{F}_{\tau}] = \psi(\tau, X_{\tau}) \end{align}
I need to show that $X$ is temporally homogeneous if and only if, for all $\tau$ and $\phi$, the above $\psi$ does not depend on $\tau$ (namely it takes the form $\psi(X_{\tau})$).
I am trying to use smoothing property of conditional expectation to solve this:
\begin{align} \mathbb{1}_{\{\tau = n\}} \; \mathbb{E}[\phi(X_{n+1}) \; | \; \mathcal{F}_n] &= \mathbb{1}_{\{\tau = n\}} \; \mathbb{E}[\phi(X_{n+1}) \; | \; X_n] \\ &= \mathbb{E}[\phi(X_{\tau+1}) \; | \; X_{\tau}] \\ &= \psi(X_{\tau}) \end{align}
I cannot figure out if I have implicitly used time homogeneity here. I also don't know how to establish the sufficiency (how time-homogeneity results from sole dependence of $\psi$ on $X_{\tau}$). I would appreciate any insights on this.
So, because $X$ is a Markov chain, there is a function $p:\Bbb N\times S\times S\to[0,1]$ such that $\Bbb P[X_{n+1}=j\,|\,\mathcal F_n] = p(n,i,j)$, a.s. on $\{X_n=i\}$, for each $(n,i,j)\in \Bbb N\times S\times S$. And $X$ is temporally homogeneous if and only if $p(n,i,j) = p(m,i,j)=p(i,j)$ for all $(i,j)\in S\times S$, for all $m$ and $n$.
Added: This answers "I need to show that $X$ is temporally homogeneous if and only if, for all $\tau$ and $\phi$, the above $\psi$ does not depend on $\tau$ (namely it takes the form $\psi(X_{\tau})$)." If $X$ is temporally homogeneous, then $\Bbb E[\phi(X_{n+1}\,|\,\mathcal F_n]=\sum_{j\in S}p(X_n,j)=:\psi(X_n)$ doesn't depend on $n$. From this it follows by conditioning on the value of $\tau$ that $\Bbb E[\phi(X_{\tau+1}\,|\,\mathcal F_\tau]=\psi(X_\tau)$.
Conversely, if $\Bbb E[\phi(X_{\tau+1}\,|\,\mathcal F_\tau]=\psi(X_\tau)$ for all $\phi$ and $\tau$, then taking $\phi=1_{\{j\}}$ and $\tau=n$ (non-random) we have $\Bbb P[X_{n+1}=j\,|\,\mathcal F_n]=\psi_j(X_n)$ for some function $\psi_j$ with values in $[0,1]$. Consequently, $\Bbb P[X_{n+1}=j\,|\,X_n=i]=\psi_j(i)$. Define $p(i,j):=\psi_j(i)$ and note that $\sum_jp(i,j)=1$. It follows that $X$ is temporally homogeneous.