Let
- $E$ be an at most countable set equipped with the discrete topology and $\mathcal E=2^E$
- $X=(X_t)_{t\ge 0}$ be a discrete Markov process with values in $(E,\mathcal E)$ and distributions $(\operatorname P_x)_{x\in E}$
If $X$ is right-continuous (with respect to each $\operatorname P_x$), then $$\lim_{t\to 0+}p_t(x,x)=1\;\;\;\text{for all }x\in E\;,\tag 1$$ where $$p_t(x,y):=\operatorname P_x\left[X_t=y\right]\;\;\;\text{for }x,y\in E\text{ and }t\ge 0\;.$$
Let $x\in E$. By $(1)$, there is some $\delta>0$ such that $$p_s(x,x)>0\;\;\;\text{for all }s\in [0,\delta]\;.\tag 2$$ Now, the Chapman–Kolmogorov equation yields $$p_t(x,x)\ge p^n_\delta(x,x)p_s(x,x)\stackrel{(2)}>0\;\;\;\text{for all }t\ge 0\;,\tag 3$$ where $$n:=\left\lfloor\frac t\delta\right\rfloor\;\;\;\text{and}\;\;\;s:=t-n\delta\;.$$
So, the probability that $X$ started in $x$ returns to $x$ after time $t$ is always positive.
That sounds weird to me. since it seems to imply that $E$ cannot contain absorbing states. But isn't
a counterexample? Is there a contradiction to the right-continuity or any other assumption?
The catch here is the word "return". You have shown that $\Bbb P_x[X_t=x]>0$ for all $t>0$, but it may be that all of the probability is due to paths that haven't yet left $x$ by time $t$. This is not incompatible with "return to $x$" (in the sense of leaving state $x$ and then coming back to $x$) having probability $0$, as in your example.