I'm not that good with math, and recently I got confused on what exactly a discrete state space means, and the difference between DTMC and CTMC.
For DTMC and CTMC, I know that it means Sx is finite or countable and normally represented by the set of integers. What does finite or countable set mean? Like if X(t) can take up values from 0 to $\infty$, is that still considered a countable set?
Thank you
A finite set is of the form $\{1,2,\cdots,r\}$ and a countable set is either a finite set or one that can be put in bijection with the natural numbers $\{1,2,3\cdots\}$.
In this context, either the state space will be finite or will be (for example) taking up values from the set $\{1,2,3,\cdots\}$. If the state space assumes values from the set $\{\cdots,-3,-2,-1,0,1,2,3,\cdots\}$ then also it is fine, since $\{\cdots,-3,-2,-1,0,1,2,3,\cdots\}$ can be put in bijection with the set $\{1,2,3\cdots\}$.