I started working on probability theory and there is something I am confused with.
Consider a transition function p on a countable and compact space $X$ that is $p(x^\prime|x)$.
How can I understand the difference between $p(dx^\prime|x,a)$ and $p(x^\prime|x,a)$? In particular do $p(dx^\prime|x,a)$ and $p(x^\prime|x,a)$ coincide for countable space?
In addition, assuming weak continuity or continuity holds on $p(dx^\prime|x,a)$, how does it apply to $p(x^\prime|x,a)$? My guess is, if it coincides then the (weak) continuity assumption applies as well.
Thank you for your help.