Markov chain notation

699 Views Asked by At

In a book of stochastic approximation, in the convergence of the ODE method chapter I see the following notation :

the state vector of a system $X_n$ has a dynamic representation controlled by $\theta$ and so,

$$P(\eta_n \in G | \eta_{n-1}, \eta_{n-2}, \dots; \theta_{n-1}, \theta_{n-2},\dots) = \int_{G} \pi_{\theta_{n-1}} (\eta_{n-1},dx)$$ where $\pi_{\theta}$ is the transition probability of a $\theta$-dependent Markov Chain $\eta_n$

1

There are 1 best solutions below

0
On

The process $(\eta_n)$ is a nonhomogenous Markov chain since the transition kernel from $\eta_{n-1}$ to $\eta_n$ is $\pi_{\theta_{n-1}}$, which may depend on the value of $\theta_{n-1}$.