Total variation distance of probaiblity measures

Question

Let $E$ be a set and $\mathcal E\subseteq 2^E$ with $\emptyset\in\mathcal E$. If $\mu:\mathcal E\to\mathbb R$ with $\mu(\emptyset)=0$, then $$\operatorname{Var}_\mu(B):=\sup\left\{\sum_{i=1}^n\left|\mu(B_i)\right|:n\in\mathbb N\text{ and }B_1,\ldots,B_n\in\mathcal B\text{ are disjoint with }\biguplus_{i=1}^nB_i\subseteq B\right\}$$ for $B\subseteq E$. If $E\in\mathcal E$, let $$\left|\mu\right|:=\operatorname{Var}_\mu(E).$$

Assume $(E,\mathcal E)$ is a measurable space and $\mu,\nu$ are probability measures on $(E,\mathcal E)$. Is it possible to show that $$\left|\mu-\nu\right|=\sup_{B\in\mathcal E}\left|\mu(B)-\nu(B)\right\|\tag1$$ or is there a counterexample?

I was able to show the claim assuming that $\mu$ and $\nu$ are both absolutely continuous with respect to a common reference measure. Is it possible to show $(1)$ in general?

Answer 1

You are off by a factor of 2: $\| \mu-\nu\| =2\sup_{B\in\mathcal E}|\mu(B)-\nu(B)|$. For this it's important that $\mu $ and $\nu$ are probabilities, so that $\mu(E)-\nu(E)=0$.

Later addition: Let $\lambda=\mu+\nu$ and let $f$ and $g$ be the Radon-Nikodym densities of $\mu$ and $\nu$ with respect to $\lambda$. It's not hard to check that $\|\mu-\nu\|=\int_E|f-g|\,d\lambda$.

First, because $\mu(E)=\nu(E)=1$, if $B\in\mathcal E$ then, $$ 2|\mu(B)-\nu(B)|=|\mu(B)-\nu(B)|+|\mu(B^c)-\nu(B^c)|\le\operatorname{Var}_{\mu-\nu}(E)=\|\mu-\nu\|. $$ This shows that $\|\mu-\nu\|\ge 2\sup_{B\in\mathcal E}|\mu(B)-\nu(B)|$.

For the reverse inequality, $$ \eqalign{ \|\mu-\nu\|&=\int_E|f-g|\,d\lambda\cr &=\int_{\{f>g\}}(f-g)\,d\lambda+\int_{\{f<g\}}(g-f)\,d\lambda\cr &=(\mu-\nu)(\{f>g\})+(\nu-\mu)(\{f<g\})\cr &=|(\mu-\nu)(\{f>g\})|+|(\mu-\nu)(\{f<g\})|\cr &\le 2\sup_{B\in\mathcal E}|\mu(B)-\nu(B)|. } $$

Answer 2

Your claim that the equality holds when the measures are absolutely continuous w.r.t. some measure is wrong. You can check that it fails even in simple cases like $\mu (A)=m(A\cap (0,1))$, $\nu (A)= m(A\cap (-1,0))$ where $m$ is Lebesgue measure. [Any two probability measures are absolutely continuous w.r.t some probability measure, e.g. $\frac {\mu +\nu} 2$].

If $\mu =\delta_a$ and $\nu =\delta_b$ with $a \neq b$ then $|\mu-\nu|=2$ and $|\mu(B)-\nu(B)| \leq 1$ for all $B$.

[$|\mu-\nu| \geq |(\mu-\nu)(\{a\})|+|(\mu-\nu)(\{b\})|=2$].

More generally if $P$ and $Q$ are any two probability measures with $P\perp Q$ the $|P-Q|=2$ and $|P(B)-Q(B)| \leq 1$ for all $B$. What is true in general is RHS $\leq$ LHS $\leq 2$RHS.