Suppose that:
- $z$ follows a discrete-time Markov process with a finite state space and transition-matrix $P$.
- $x$ follows $x'=g(x,z)$, where $x'$ denotes next-period $x$.
- $g(\cdot,z)$ is continuously differentiable and invertible, for every $z$.
- $y$ follows $y'=a(x,z)+b(x,z)y.$
- $z$ takes values in $\{z_1,\dots,z_n\}$, while $x$ and $y$ take values in the real line.
I need to find a recursive formula for $\mathbb E[y\mid x,z]$.
As a first step, one could apply the conditional expectation to both sides of the recursive equation for $y$, which yields $$\mathbb E[y'\mid x,z] =a(x,z)+b(x,z)\mathbb E[y\mid x,z]. $$ I am guessing that the Markov structure of this system should allow for some link between $\mathbb E[y'\mid x,z]$ and $\mathbb E[y'\mid x',z']$ which would then lead to a recursive formula, but I haven't been able to find it.
Edit:
It seems to me that the Markov structure implies that: $$ \mathbb E[y'\mid x',z']=\sum_{z}P_{z,z'}\mathbb E[y'\mid g^{-1}(x',z),z].$$ I don't know how to formally derive this equation though, so I'm not sure it is correct.
Here is a partial answer with some progress. Let us use $x_t,z_t,y_t$ to denote the "current values," and $x_{t+1},z_{t+1},y_{t+1}$ to denote the "next" ones. Consider an intermediate conditional expectation with $z_t$ added:
\begin{equation*} \mathbb{E}\left(y_{t+1}|x_{t+1},z_{t+1},z_t\right) = a\left(g^{-1}\left(x_{t+1},z_t\right),z_t\right) + b\left(g^{-1}\left(x_{t+1},z_t\right),z_t\right)\mathbb{E}\left(y_t|x_{t+1},z_{t+1},z_t\right) \end{equation*}
Now, observe that $y_t$ depends on $z_{t-1},z_{t-2},...$. The Markov property works backward in time, so $z_{t+1}$ can be dropped from the condition, since $z_t$ is the "earliest" future value. Then,
$$\mathbb{E}\left(y_t|x_{t+1}=x',z_t\right) = \mathbb{E}\left(y_t|x_{t}=g^{-1}\left(x',z_t\right),z_t\right)$$
so if you already have a form for $\mathbb{E}\left(y_t|x_t,z_t\right)$, you can indeed use the above transformation, which is similar to what you have written.
The problem is that $z_t$ is not really given. You can write
$$\mathbb{E}\left(y_{t+1}|x_{t+1},z_{t+1}\right) = \sum_k \mathbb{E}\left(y_{t+1}|x_{t+1},z_{t+1},z_t=k\right)P\left(z_t=k|x_{t+1},z_{t+1}\right)$$
but the issue is that the backward transition probabilities are nonstationary, and it is not the case that $P\left(z_t =k|z_{t+1}=j\right) = P_{kj}$. These probabilities can be simplified if the chain is time-reversible or is initialized using its invariant distribution. The presence of $x_{t+1}$ in the condition also complicates matters.