Higher order developing Markov chains?

Question

Are there Markov chains, possibly conventionally defined higher order Markov chains, whose state space is developing, I.e. number of states are changing and transition matrix are changing, e. G., by application some dynamic operator to it. Maybe such changes can be described by other, "higher order" Markov chain itself. Are there such constructions and can they be used to describe dynamic environment in reinforcement learning. Or any such concept can be expressed by extremely large, augmented Markov chain? Howeve compact representation, special structures and consciousness could warrant such notion?