We know that ginen a (discrete time) markov chain $\{X_n\}$, its transition matrix is stochastic. Is the inverse true? I.e. given a stochastic matrix $P$, is there a (discrete time) markov chain $\{X_n\}$ having $P$ as its transition matrix?
Attempt. I believe so. We are dealing with an existence theorem, and my idea has been the use of Kolmogorov's extension theorem, since it guarantees the existance of a desired markov chain under certain properties. Am I on the right path?
Thanks for the help!