I have seen discrete-time discrete-state Markov processes (such as random walks), continuous-time discrete-state Markov processes (such as Poisson processes), and continuous-time continuous-state Markov processes (such as Brownian motions).
I was wondering if discrete-time continuous-state Markov processes have been studied as often as the above three?
What are some of its examples then?
Discrete-time continuous state Markov processes are widely used. Autoregressive processes are a very important example.
Actually, if you relax the Markov property and look at discrete-time continuous state stochastic processes in general, then this is the topic of study of a huge part of Time series analysis and signal processing.
The most famous examples are ARMA processes, the Conditionally Heteroscedastic models, a large subclass of Hidden Markov models....