In reading about Wasserstein GANS here, an example is given in the middle of page $4$.
Let $Z \sim U [0,1]$, and let $P_0 \sim [0,Z] \in \mathbb{R}^2$. Let $P_{\theta} := (\theta, Z)$ be a probability distribution parametrized by a constant $\theta$.
The following identity is stated but not argued:
- The Wasserstein $1$-distance (also referred to as the Earth Mover's Distance) $W$ between $P_0$ and $P_\theta$ satisfies $W(P_0, P_\theta) = | \theta|$
For the Wasserstein $1$-distance, I am accustomed to using this metric for probability distributions of one variable, so in this case I think (not sure) that $$W(P_0, P_\theta) = \inf_{\gamma \in \Pi (P_0, P_{\theta} )}\int_{x \in X} \int_{x' \in X'} \int_{y \in Y} \int_{y' \in Y'}\|(x,y) -(x', y')\| \gamma(x,y, x', y')\,\mathrm{d}x \mathrm{d}x' \mathrm{d}y \mathrm{d}y'$$ where $\Pi(P_0, P_{\theta} )$ is the set of joint distributions of $(x,y)$ with marginals equal to $P_0$ and $P_{\theta}$.
Consider two cases. If $\theta = 0$, then the distributions are equal and the distance is $0$. Now suppose that $\theta \neq 0$, it seems the first step to compute the desired integral is to reason what is $\inf_{\gamma \in \Pi (P_0, P_{\theta} )} $?
Since the first co-ordinate of both distributions is constant for $(x,y)$ pairs with non-zero support, then it seems like $\gamma(x, y, x', y')$ has two cases. If $x=0, x'= \theta$, then for $(y,y') \in [0,1] \times [0,1]$ we have $\gamma(x,y,x',y') = 1$, and $\gamma$ is zero otherwise.
How to use the above to show that $W(P_0, P_\theta) = |\theta|$ eludes me, any insights appreciated.
To simplify the notation a bit we note that $W(P_0,P_\theta)$ is given by $\inf\mathbb{E}[\|(0,Z)-(\theta,Z')\|]$, where the infimum is taken over all distributions of $(Z,Z')$ such that the marginals are $U(0,1)$-distributed. Regardless of the joint distribution of $(Z,Z')$ it holds that $$ \mathbb{E}[\|(0,Z)-(\theta,Z')\|]\geq\mathbb{E}[\|(0,Z)-(\theta,Z)\|]. $$ This shows that the infimum is attained at the joint distribution $(Z,Z')\sim(Z,Z)$. Hence, $$ W(P_0,P_\theta)=\mathbb{E}[\|(0,Z)-(\theta,Z)\|]=|\theta|. $$