I need advice from experienced mathematicians with regards to the following setup and notation.
Suppose $(X_t)_{t=0}^T$ is a controlled Markov process where random variables $X_t:\Omega \to R^n$ for $t =0,\dots, T$. Suppose this processes is controlled by a sequence of actions $(a_t)_{t=0}^{T-1}$ where $a_t$ affects the evolution from $X_t$ to $X_{t+1}$.
Is it wrong that I express this evolution as $$ X_{t+1} = f_{t+1}(Y^{a_t}_{t+1}, X_t) $$ where $Y^{a_t}_{t+1}$ are random variables driving the controlled Markov process through the transition function $f_{t+1}$???
I showed this to my teacher and he said it is nonsense because the random variables $Y^{a_t}_{t+1}$ cannot be action dependent. He gave an explanation but I still didn't quite understand why. So I've been trying to convince myself this.
I looked through some textbooks and indeed, the random variables seem not to be action dependent. For example, a lot of people use something like $$ X_{t+1} = f_{t+1}(a_t, Y_{t+1}, X_t) $$ instead. But I still don't understand what's wrong with using the first equation above.
Does anyone understand?