What does $\mathbb{P}(X_1=x_1,\ldots,X_n=x_n,\sum_{i=1}^n X_i = t)$ actually mean?

Question

Let $X = (X_1,X_2,\ldots,X_n)$ be a vector of iid Bernoulli random variables with parameter $\theta$.
We want to show that $T(X):= \sum_{i=1}^n X_i$ is a sufficient statistic for $\theta$.

This is what they did:

$$P\left(X=x\mid \sum_{i=1}^n X_i = t\right) = \frac{P(X_1 = x_1,X_2=x_2,\ldots,X_n=x_n,\sum_{i=1}^n X_i = t)}{P(\sum_{i=1}^n X_i=t)} \\ =\frac{\theta^{\sum_{i=1}^n x_i}(1-\theta)^{n-\sum_{i=1}^n x_i}}{\binom{n}t \theta^t (1-\theta)^{n-t}} = \frac{1}{\binom{n}t}.$$

What I don't understand is from the first equality to the second, it just looks like they used the fact that
$$P(X_1=x_1,\ldots,X_n=x_n) = P(X_1=x_1)\ldots P(X_n=x_n) = \theta^{\sum_{i=1}^n x_i} (1-\theta)^{n-\sum_{i=1}^n x_i} $$ ... so it seems that $$P(X_1=x_1,\ldots,X_n=x_n) = P(X_1=x_1)\ldots P(X_n=x_n) = P(X_1=x_1,\ldots,X_n=x_n) = P(X_1=x_1,\ldots ,X_n=x_n,\sum_{i=1}^n X_i = t) $$
Can someone please explain this? Shouldn't there be another factor of $$P\left(\sum_{i=1}^n X_i = t\right)$$
by independence as well?

Answer 1

By $$P\left(X_1=x_1,\ldots ,X_n=x_n,\sum_iX_i=t\right) $$ they do mean what it looks like: the probability that $X_1=x_1$ and $X_2=x_2,$ etc and that also $\sum_iX_i=t.$ And they do write the correct expression in the numerator except that there should be a constraint that $\sum_i x_i = t.$ So we can replace the occurrences of $\sum_i x_i$ with $t,$ which is how they get to the final expression (which should also have a constraint on it). To be explicit, we have $$P\left(X_1=x_1,\ldots ,X_n=x_n,\sum_iX_i=t\right) = \theta^{\sum_i x_i}(1-\theta)^{n-\sum_i x_i}1_{\sum_i x_i=t}= \theta^t(1-\theta)^{n-t}1_{\sum_i x_i=t}.$$

The reasoning behind this expression is that of course if $\sum_i X_i = t$ we are going to have zero unless $\sum_i x_i = t.$ And on the other hand, if we do meet that constraint, then the $\sum_i X_i=t$ is redundant, so we just have the same thing as $P(X_1=x_1,\ldots,X_n=x_n).$

Answer 2

Generally the notation there is deceptive: the numerator in the second line does involve exponents of $\sum_i x_i$ and $n-=\sum_i x_i$, but these are supposed to be assumed to be $t$ and $n-t$ respectively.

The probability of a given configuration which has sum $t$ is $\theta^t (1-\theta)^{n-t}$: you have $t$ factors of $\theta$ and $n-t$ factors of $(1-\theta)$ in the multiplication of the probabilities. The probability of having sum $t$ in total is given by the binomial PMF. As you know, each of these configurations with a particular sum has the same probability, so the binomial PMF is just ${n \choose t}$ times the probability of each configuration. Thus the probability of each configuration cancels out, and so the conditional probability can be written in the usual "number of ways for X to happen divided by number of elements in sample space" form.

The main point of this calculation from the point of view of statistical inference is that once you know the sum, you cannot discern any more information about $\theta$ by inspecting the actual configuration, because conditioned on the value of the sum, the probabilities of each possible configuration are all the same.

Answer 3

First to show sufficient statistics you want to show the probability you're calculating doesn't depend on $\theta$. This goal is achieved in the end, as you can see in the end. To answer your question, you can see that $t$ is redundant since if you have all values of $x$ then their sum is also know. So $P(X=x,\sum X=t)=P(X=x)$.

Answer 4

The statement $X_1 = x_1,X_2=x_2,\ldots,X_n=x_n,\sum_{i=1}^n X_i = t$ is redundant and means the same thing as $X_1 = x_1,X_2=x_2,\ldots,X_n=x_n.$ And then you use independence of the $n$ random variables to write that as a product: $$ \Pr(X_1=x_1) = \theta^{x_1} (1-\theta)^{1-x_1} = \begin{cases} \theta & \text{if } x_1=1, \\ 1-\theta & \text{if } x_1=0. \end{cases} $$ Then multiply and get $\theta^{x_1+\cdots+x_n} (1-\theta)^{n-(x_1+\cdots+x_n)}.$