$f_1,f_2,f_3: \mathbb R\to \mathbb R$, starting from a deterministic $X_0=x\in \mathbb R$, let $X_1=f_1(X_0)$ with probability $p_1=1/3$, $X_1=f_2(X_0)$ with probability $p_2=1/6$, $X_1=f_3(X_0)$ with probability $p_3=1/2$.
I observe $X_1,\dots, X_{10}$.
Now, starting from $X_{10}$ which I understand not deterministic, let $X_{11}=f_1(X_{10})$ with probability $q_1=1/2$, $X_{11}=f_2(X_{10})$ with probability $q_2=1/5$, $X_{11}=f_3(X_{10})$ with probability $q_3=3/10$.
I see that $\mathbb E[X_1\mid X_0=x]=\sum\limits_{k=1}^{3}p_kf_k(x)$? I am not sure, as after every 10 steps $p_i$ changes to $q_i$..making me confuse about the expectation. Thanks for any help.
Well, I mean, the situation was always going to be a little complicated. Let's define $f \circ g(x) = f(g(x))$ (the composition of functions). Let's first try to work out the situation when $X_0$ is given, then go for the general $\mathbb E[f(X_n) | X_{n-k}]$ for $k \leq n$.
First, $\mathbb E[f(X_1) | X_0 =a]$. That is $\sum_{k=1}^3 p_k f(f_k(a))$, since $X_1$ is $f_i(a)$ with probability $p_i$, $i=1,2,3$.
Now what about $\mathbb E[X_2 | X_0 = a]$? Well, for $i,j = 1,2,3$ , with probability $p_ip_j$, we have $X_2 = (f_i \circ f_j)(a)$. Hence, $$ \mathbb E[X_2 | X_0 = a] = \sum_{i,j = 1}^3 p_ip_j( f \circ f_i \circ f_j)(a) $$
in similar fashion, you can see that $$ \mathbb E[X_{10} | X_0 = a] = \sum_{k_1,...,k_{10} = 1}^3 p_{k_1}...p_{k_{10}} f \circ f_{k_1} \circ ... f_{k_{10}}(a) $$
once we go to a number $11 \leq n \leq 20$ we have to add the $q_{i}$ into play. But it is easy to see that for example : $$ \mathbb E[f(X_{17}) | X_0 = a] = \sum_{k_1,...,k_{17} = 1}^{3} q_{k_{17}}...q_{k_{11}}p_{k_{10}} ... p_{k_1} (f \circ f_{k_{17}} \circ ... \circ f_{k_1})(a) $$
now what happens if we go to a very large quantity? Say $n = 46$. Then for $1 \to 10, 21 \to 30 , 41 \to 46$ we use $p$, for $11 \to 20, 31 \to 40$ we use $q$. So , we have : $$ \mathbb E[X_{46} | X_{0} = a] = \sum_{k_{1},...,k_{46} = 1}^{3} p_{k_{46}}p_{k_{45}}...p_{k_{41}}p_{k_{30}}...p_{k_{21}}p_{k_{10}}...p_{k_1} q_{k_{40}}...q_{k_{31}}q_{k_{20}}...q_{k_{11}} (f \circ f_{k_{46}} \circ ... \circ f_{k_1})(a) $$
so we are slowly warming to our task. In particular, let us see how this works. What does not change is the composition of functions, which has a clear pattern. Instead, we need to look at when $p$ is used and when $q$ is used. Then again, isn't that pattern clear? See the remainders of the numbers which sit beneath a $p$ when divided by $20$ : it's all between $1$ and $10$. On the other hand, the numbers which sit beneath a $q$ have remainders between $11$ and $19$, and $0$. Let $i \% 20$ be the remainder upon division by $20$ (note : the percentage sign may seem unusuual, but in a lot of programming languages it is considered standard notation for the remainder), and let $P = \{1,2,...,10\}$ and $Q= \{0,11,12,...,19\}$. Then we have the general expression :
$$ \mathbb E[f(X_n) | X_0 = a] = \sum_{k_i=1 , 1 \leq i \leq n}^3 \left(\prod_{1 \leq j \leq n, j \% 20 \in P} p_{k_j}\right) \left(\prod_{1 \leq l \leq n, l \% 20 \in Q} q_{k_l}\right) (f \circ f_{k_{n}} \circ ... \circ f_{k_1})(a) $$
Generalizing this is not going to be very difficult : indeed, once we have the remainders written down we are in great shape.
For example, let's take $\mathbb E[X_{54} | X_{16} = a]$. Then for the transitions at times $21 \to 30,41 \to 50$ we have to use $p$ and $17,18,19,20,31\to 40, 51 \to 54$ we have to use $q$. With this in mind, when we index $l_1,...,l_{k}$ in the sum, then we must actually look at whether $(n-k)+1, (n-k)+2,...,(n-k)+k$ belong in $P$ or $Q$ and perform accordingly.
That kind of analysis leads to the following expression for $n \geq k$ : $$ \mathbb E[f(X_{n}) | X_{n-k}] = \sum_{l_i = 1 , 1 \leq i \leq k}^3 \left(\prod_{1 \leq j \leq k , ((n-k)+j) \% 20 \in P} p_{l_{j}}\right)\left(\prod_{1 \leq m \leq k , ((n-k)+m) \% 20 \in Q} q_{l_{m}}\right)(f \circ f_{l_k} ... \circ f_{l_1})(a) $$
In general, this is a difficult expression to work with because compositions of functions are difficult to understand. However, this is comfortably the most general one can get.
Note : one can write membership in $P,Q$ using the greatest integer function. For $N$ let $L = \left[\frac N{20}\right]$. Then $N - 20L = N \% 20$. With that, we note that $N \% 20 \in P$ means that $1 \leq N - 20\left[\frac N{20}\right] \leq 10$, and if $N \% 20 \in Q$ then note that $10 \leq N - 20 \left[\frac N{20}\right] \leq 19$ or $ N - 20 \left[\frac N{20}\right] = 0$. Thus we can write the conditions that way as well.