Toy dice problem and conditioning

Question

Let $X_1$ and $X_2$ be the values of two fair dice (with 6 faces). For the sake of understanding (and quantifying) how conditioning on a particular sum of the dice modifies the joint distribution of $X_1$ and $X_2$, can you confirm or correct the following total variation computation: $$2\|Pr((X_1, X_2) | X_1 + X_2 = 6) - Pr((X_1, X_2))\|_{VAR}$$ $$= \sum_{\substack{(x_1,x_2) \in [6]\times[6]}\\x_1 + x_2 = 6} \left| Pr(X_1 = x_1, X_2 = x_2) - Pr(X_1 = x_1, X_2 = x_2) \right|$$ $$+ \sum_{\substack{(x_1,x_2) \in [6]\times[6]}\\x_1 + x_2 \neq 6} \left| Pr(X_1 = x_1, X_2 = x_2) \right|$$ $$ = \sum_{\substack{(x_1,x_2) \in [6]\times[6]}\\x_1 + x_2 \neq 6} \left| Pr(X_1 = x_1, X_2 = x_2) \right| $$ $$= \frac{1}{36} \left| \left\{ (x_1,x_2) \in [6]\times[6]: x_1 + x_2 \neq 6 \right\} \right| = \frac{31}{36}$$ Is this correct/rigorous ?

---

After the answers of астон вілла олоф мэллбэрг and Rocco:

$$2\|Pr((X_1, X_2) | X_1 + X_2 = 6) - Pr((X_1, X_2))\|_{VAR}$$ $$= \sum_{\substack{(x_1,x_2) \in [6]\times[6]}\\x_1 + x_2 = 6} \left| \frac{Pr(X_1 = x_1, X_2 = x_2 \text{ and } X_1 + X_2 = 6)}{Pr(X_1 + X_2 = 6)} - Pr(X_1 = x_1, X_2 = x_2) \right|$$ $$+ \sum_{\substack{(x_1,x_2) \in [6]\times[6]}\\x_1 + x_2 \neq 6} \left| Pr(X_1 = x_1, X_2 = x_2) \right|$$ $$ = 5 * |(1/36)/(5/36) - 1/36| + \frac{1}{36} \left| \left\{ (x_1,x_2) \in [6]\times[6]: x_1 + x_2 \neq 6 \right\} \right| $$ $$= \frac{31}{18}$$

Answer 1

There is something wrong in the first summation, I will explain.

Recall that the Total Variation distance is between two measures.

The second measure is the uniform measure on the sample space $\Omega = \{1,...,6\}^2$ i.e. $G(\omega) = \frac 1{36}$ for all $\omega \in \Omega$.

The measure $Pr((X_1,X_2) | X_1 + X_2 = 6)$ on the sample space $\Omega$ is more different. Naturally, it assigns $0$ to all points which don't sum to $6$. What if they do? Then the points $(1,5),(2,4),(3,3),(4,2)$ and $(5,1)$ remain, which get assigned equal probabilities, namely $0.2$ each. So, the description here is that $F(\omega) = 0.2$ if $\omega$ belongs to that set summing to $6$, else $F(\omega) = 0$. For more details on similar computations, see here.

Now if we do $2\|F-G\|_{TV} = \sum_{\Omega} |F(\omega) - G(\omega)|$, this breaks into two sums, one for which the sum is not six, the other for which the sum is six.

Those for which the sum is not six, yes you are right, this case in covered in the third line of your computation.

For the other case, you are incorrect! So your second line is not zero, but rather is $5 \times \left|0.2 - \frac 1{36}\right|$, which equals $\frac {31}{36}$, the same as the other computation! Therefore, the answer is $2||F-G||_{TV} = \frac{31}{18}$.

---

Why is the answer for both computations equal to $\frac {31}{36}$? That's because of the following observation : recall that the total variation distance is also the supremum over all sets $A$ of $|F(A) - G(A)|$ where $F,G$ are the prob. measures in question.

Indeed, if I just take the set $B = \{x : F(x) \leq G(x)\}$, then one checks that $|F(B) - G(B)| = \frac{31}{36} = \sum_{B} [F(x) - G(x)] $, and also that $\sum_{B} [G(x) -F(x)] = \sum_{B^c} [F(x) - G(x)] = \frac {31}{36}$. Now, you had got the first term correct, but the second term you wrote zero for it, so the answer was exactly half of what you got.

EDIT : This last insight also forms part of a proof of the equivalence of the two expressions of TV.

Answer 2

Ok, I think there is an error there!

First, a qualitative line of reasoning to spot it; then, I will point at the computation error:

Qualitative:

Call $\delta P$ your total variation. Your final result is then $2\delta P = 31/36$, giving: $$ \delta P = \frac{31}{72}$$

As the total variation between two distributions $P$ and $Q$ is defined as ($\mathcal F$ is your $\sigma$-algebra of events): $$\delta P = \sup_{A\in\mathcal F} |P(A)-Q(A)| $$ to prove your result wrong it is sufficient to find a set $A^*$ such that $|P(A)-Q(A)|\gt \frac{31}{72}$.

You can check that $A^*=\{(1,5),(2,4),(3,3),(4,2),(5,1)\}$ does the job: $$P_{X,Y|X+Y=6}(A^*)-P_{X,Y}(A^*) = 5\cdot \frac{1}{5} - 5\cdot \frac{1}{36}= 1-\frac{5}{36}=\frac{31}{36}\gt \frac{31}{72}$$

Quantitative:

Now, what was the error in your computation? In the second line you write the sum over the $(x_1+x_2=6)$ set in a way that makes you think it gives 0. But the two probabilities that you are subtracting in each term of that sum are not equal! In fact, the one relative to the conditioned distribution has value $\frac{1}{5}$ for each term of the sum, while the other is the usual $\frac{1}{36}$. If you take into account those corrected terms, you will double your final result, and get: $$2\delta P = 31/18$$ leading to the correct result $$\delta P = \frac{31}{36}$$