If we don't know the transition probability, then how to prove a markov chain is recurrent or irreducible?

36 Views Asked by At

For example, we only have the next state information: $q(n+1) = q(n) + a - c$, where $a$ can be regarded as some arrival and $c$ can be regarded as some action. Also, $a,c \in \{0,1,\cdots\}$, which follows independently and identically distributed.

Can any friends give some comments on how to prove it?

1

There are 1 best solutions below

2
On BEST ANSWER

Assuming that $E|a-c|<\infty$, the proposed chain is recurrent if and only if $E[a-c] =0$. See, e.g.

Spitzer, Frank. Principles of random walk. Vol. 34. Springer Science & Business Media, 2013.