Fix a probability space $(\Omega, \mathcal{F})$. Let $P$ and $Q$ be two probability measures on $(\Omega, \mathcal{F})$ such that there exist $\varepsilon$ and $\delta$ that for every $A \in \mathcal{F}$ we have $P(A) \leq e^\varepsilon Q(A) + \delta$ and $Q(A) \leq e^\varepsilon P(A) + \delta$.
Question: Is the following claim true?
There exists probability measures $\mu$, $\mu'$, $\nu$, $\nu'$ and a constant $C >0$ such that the following holds:
- $P = (1-\delta)\mu + \delta \mu'$
- $Q = (1-\delta)\nu + \delta \nu'$
- $\Vert\frac{\text{d}\mu}{\text{d}\nu} \Vert_{\infty} \leq C$ (ess-sup with respect to $\nu$)
- $\Vert \frac{\text{d}\nu}{\text{d}\mu} \Vert_{\infty} \leq C$ (ess-sup with respect to $\mu$)
For the case of $\varepsilon=0$, this claim is true and the proof is based on the relationship between total variation distance and Wasserstein distance.
Let $\tau=1/2 P+1/2Q$. Then both $P$ and $Q$ are absolutely continuous with respect to $\tau$ and admit densities $p$ and $q$, respectively, and the total variation distance between $P$ and $Q$ is given by $\int|p-q|~\mathrm d\tau$. Now, $$p=p\wedge q+(p-q)_+$$ and $$q=p\wedge q+(q-p)_+$$ and $\int (p-q)_+~\mathrm d\tau=\int (q-p)_+~\mathrm d\tau=\delta/2$ and $\int p\wedge q~\mathrm d\tau=1-\delta/2$. Let $$g=\frac{1}{1-\delta/2}~ p\wedge q$$ and $$h=\frac{1}{\delta/2}~ (p-q)_+.$$ Then $$p=(1-\delta/2)g+\delta/2~h=(1-\delta)g+\delta/2~g+\delta/2~h=(1-\delta)g+\delta(1/2 g + 1/2 h).$$ Note that $g$ and $h$ are densities with respect to $\tau$. We can then take $\mu$ to be the probability measure that has density $g$ with respect to $\tau$ and $\mu'$ be the probability measure that has density $1/2~g+1/2~h$ with respect to $\tau$.
A parallel argument works for $Q$ and we can even take $\nu=\mu$.