Is {$X_n,n\geq 0$} a markov chain?

1.6k Views Asked by At

Consider a process {$X_n,n=0,1,\dots$}, which takes on the values $0,1,2$. Suppose $$P(X_{n+1}=j|X_n=i,X_{n-1}=i_{n-1},\dots,X_0=i_0)$$ $$=P_{ij}^I,\text{when n is even}$$ $$=P_{ij}^{II},\text{when n is odd}$$ where $\sum_{j=0}^2P_{ij}^I=\sum_{j=0}^2P_{ij}^{II}=1,i=0,1,2$. Is {$X_n,n\geq 0$} a markov chain? If not then show how, by enlarging the state space, we may transform it into a markov chain.

For verify if is a Markov Chain I just need to show that $$(1)P_{ij}\geq 0 \text{ and}(2)\sum_{j=0}^\infty P_{ij}=1,i=0,1,2\dots$$ In this case I think there is three states $i=0,1,2$. I think $(2)$ is already satisfied by what was given in the question, but as I show $(1)$?

Maybe my ideas are wrong, and is not what I have to show.

1

There are 1 best solutions below

3
On BEST ANSWER

A non-autonomous process is not Markov, basically because the distribution of $X_{n+1}$ cannot be specified solely by specifying the distribution of $X_n$. Try to prove this.

On the other hand, a process which is "only not Markov because it is not autonomous" (i.e. where the distribution of $X_{n+1}$ can be specified from $n$ and $X_n$ but not from $X_n$ alone) can be made Markov by considering the process $Y_n=(n,X_n)$. This turns out to be a rather general phenomenon: we can reduce the dimension at the cost of memory effects, or we can remove memory effects by increasing the dimension.

Note that in theory what I just wrote would be a Markov process on a countably infinite state space. But because your process only cares whether the time is even or odd, you can instead enlarge the state space only to $\{ 0,1 \} \times \{ 0,1,2 \}$, where $0$ encodes even times and $1$ encodes odd times.