It seems like every discussion of reversible Markov chains assumes that the measure is invariant. A reversed chain has the transition probabilities $p'$ satisfying
$$ p'_{xy} = \frac{\pi(y)}{\pi(x)} p_{yx} $$
where $\pi$ is a stationary/invariant measure.
So is a reversed chain only defined with respect to an invariant measure? Since the Markov property holds in both time-directions, it seems to me that any Markov chain would be ``reversible". What's wrong with this point of view?
Furthermore is an irreducible chain always reversible with respect to its invariant measure?
Thanks!
A chain that is reversible with respect to $\pi$ also leaves $\pi$ invariant, as you can see by a direct calculation.
Also, the reversal procedure only makes sense when the chain is in its invariant distribution anyway. In this respect, it's true you can time reverse any chain that's already at equilibrium, but if the chain isn't reversible then you get a different chain when you do that.
Reversibility is an extremely strong assumption, it is very much not true that every ergodic chain is reversible. As a simple example:
$$\begin{bmatrix} 0.01 & 0.99 & 0 \\ 0 & 0.01 & 0.99 \\ 0.99 & 0 & 0.01 \end{bmatrix}$$
This is "essentially" the $1 \to 2 \to 3 \to 1 \dots$ cycle, with some holding time to make it ergodic (you could remove that too, I'm just covering the "what if it's aperiodic?" question at the same time). The reversed chain is instead essentially the $3 \to 2 \to 1 \to 3 \dots$ cycle (so it goes around the loop in the opposite direction).