Let
- $(\Omega,\mathcal A,\operatorname P)$ be a probability space
- $I=\mathbb N_0$ or $I=[0,\infty)$
- $(E,\mathcal E)$ be a measurable space
- $\mu$ and $\nu$ be probability measures on $(E,\mathcal E)$
- $X$ be an $(E,\mathcal E)$-valued time-homogeneous Markov chain on $(\Omega,\mathcal A,\operatorname P)$ with transition semigroup $(\kappa_t)_{t\in I}$
Assume $\mu$ is invariant with respect to $(\kappa_t)_{t\in I}$ and $$\left|\mu-\nu\kappa_t\right|\xrightarrow{t\to\infty}0\tag1,$$ where the left-hand side denotes the total variation distance of $\mu$ and $\nu\kappa_t$ and $\nu\kappa_t$ denotes the composition of $\nu$ and $\kappa_t$.
I want to show that there is a version $X^{(\eta)}$ of $X$ with $\operatorname P\left[X^{(\eta)}_0\in\;\cdot\;\right]=\eta$ for $\eta\in\left\{\mu,\nu\right\}$ with $$\tau:=\inf\left\{t\in I:X^{(\mu)}_s=X^{(\nu)}_s\text{ for all }s\in I\text{ with }s\ge t\right\}<\infty\;\;\;\operatorname P\text{-almost surely}.\tag2$$
As pointed out by E-A, the basic idea is the following: For any $t\ge0$, there is (see Theorem 2.12) a probability measure $\eta_t$ (called a coupling of $\mu$ and $\nu\kappa_t$) on $(E\times E,\mathcal E\otimes\mathcal E)$ with $$\eta_t(B\times E)=\mu(B)\text{ and }\eta_t(E\times B)=(\nu\kappa_t)(B)\;\;\;\text{for all }B\in\mathcal E\tag3$$ and $$\left|\mu-\nu\kappa_t\right|=\eta_t(\Delta^c),\tag4$$ where $$\Delta:=\left\{(x,x):x\in E\right\}.$$
Clearly, if we would be able to show (are we?) that there is a $(\mathcal A,\mathcal E\otimes\mathcal E)$-measurable $Y_t:\Omega\to E\times E$ with distribution $\eta_t$ under $\operatorname P$, we would obtain $$\eta_t(\Delta^c)=\operatorname P\left[(Y_t)_1\ne(Y_t)_2\right]\tag5.$$ However, even when we can show this, why can we choose $(Y_t)_{t\ge0}$ such that $X^{(\mu)}:=((Y_t)_1)_{t\ge0}$ and $X^{(\nu)}:=((Y_t)_2)_{t\ge0}$ are versions of $X$?
Let me try this:
Coupling $\rightarrow$ TV goes to 0:
We can argue that $P(\tau > t) \geq P(X_t \not = Y_t)$, since if $X_t$ is not equal to $Y_t$, then $\tau$ has to be greater than t. We also know that $P(X_t \not = Y_t)$ is an upper bound on the TV (or rather, the TV is a lower bound on this quantity), since the $P(X_n \not =Y_n) \geq | \mu(A) - \nu(A) | $ for any $A$. (You can find the proof in a lot of sources; if I were to summarize it, you should write $P(X_t \in A) = P(X_t = Y_t, X_t \in A) + P(X_t \not = Y_t, Y_t \in A)$, and do the same for $Y_t$; the first terms will cancel. http://websites.math.leidenuniv.nl/probability/lecturenotes/CouplingLectures.pdf; you can also graphically observe what the total variation distance is by drawing densities together and you note that it is the half of the not-overlapping area, and that gives you a pictorial proof of this statement, since you know that you can't set $X_t = Y_t$ in any of those areas.)
TV goes to 0 $\rightarrow$ coupling:
Let us construct our coupling in the most natural sense: let $X_t = Y_t$ once $X_{t_0} = Y_{t_0}$ for some $t_0$. We need to start by arguing for $P(X_t \not = Y_t | X_0, Y_0) \leq 1-\epsilon$ for some $t, \epsilon$. Now, we know that we can construct a coupling such that $P(X_t \not = Y_t) = TV(\mu, \nu_t)$ for some fixed $t$. (Proof is in the pdf posted above; Theorem 2.12; you can also argue it from the picture again). Since the TV is going to 0, you can find a $t, \epsilon$ such that the claim is true. Now, you can repeat the same argument and invoke the time homogeneity of your Markov process to show that $P(X_{2t} \not = Y_{2t} | X_t \not = Y_t) \leq 1 - \epsilon$. Now, you can inductively show a geometric decay $P(X_{2t} \not = Y_{2t}) \leq (1 - \epsilon)^k$, which is enough to show that this time is almost surely finite. (Adapted this proof from https://www2.cs.duke.edu/courses/fall15/compsci590.4/slides/lec5.pdf; Section 4, Theorem 1, I might have messed up in certain places)
Feel free to point things out if there is anything wrong/unclear/unjustified. The first one is I think OK (it is a fairly standard proof); the second one involves constructing the coupling, so not sure if all of my steps were completely justified)