In Elements of Information Theory (Thomas, Cover; 2012) they state that, given three random variables $X, Y, Z$ that form a Markov chain
$$X \to Y \to Z \iff p(x) p(y|x)p(z|y)$$
Then $X \to Y \to Z \implies Z \to Y \to X$, and consequently the chain is sometimes written as $X \leftrightarrow Y \leftrightarrow Z$.
Why is this, and is it really common to use either one of these notations interchangeably?
Personally, I have not seen the latter notation used for Markov chains very frequently. I suspect it may not be very common because of the fact that Markov chains are often considered to have some natural order; that is, we often use them in a context where it makes sense to say that $X$, $Y$ and $Z$ are "sequential in time", in which case the notation $X \to Y \to Z$ accurately conveys the idea that $X$ is "coming before" and "influencing" $Y$ which in turn "influences" $Z$.
Indeed I would refrain from using $X \leftrightarrow Y \leftrightarrow Z$ as notation, for the reason that it would seem to imply that $X \leftrightarrow Y \leftrightarrow Z \iff X \leftrightarrow Z \leftrightarrow Y$, which is not true in general.
It's not too difficult to see that $X \to Y \to Z \implies Z \to Y \to X$ is true, particularly given the equivalence you have copied there, so I'll leave that to you (if you need it is covered in this answer).