If ${\bf X}$ is multinomially distributed, then how is $Z_j = \sum_{i} \chi \{X_k = j\}$ distributed?

Question

My question is pretty easy to state, but I can't make any progress on it. Suppose that ${\bf X}$ is multinomially distributed, i.e., ${\bf X} = [X_1,\dots,X_K]\sim \text{Mult}(N,{\bf p})$ such that $\sum_{k=1}^d X_k = N$ for ${\bf p}\in R_+^d$ and $\sum_{i=1}^d p_i = 1$. Then my question is how is $Z_j = \sum_{k=1}^K \chi \{X_k = j\}$ distributed?

This new random variables tells us how many of the entries of ${\bf X}$ are equal to $j$.

[Note that the $\chi$ is meant to represent the indicator function.]

To make things easier, if necessary, we can assume that the vector of probabilities for the multinomial distribution has equal entries, i.e., $p_i = 1/d$ for all $i=1,\dots,d$.

Best, Jake

Answer 1

Let $\{X_1, \dots, X_n\}$ be a collection of $n$ i.i.d. multinomial random variables. Then $p_{j} := \Pr(X_1 = j) = \cdots = \Pr(X_n = j)$. Then,

$\Pr(Z_j = k) = {n \choose k}p_j^k (1 - p_j)^{n-k}$. Plug in the p.d.f. of $p_j$ to get the final distribution.

Answer 2

To expand on @rodms answer, and explain what is going on, it might be useful to forget about probabilities for a minute.

The multinomial distribution can be seen as a tool to study processes that partition a finite set into a fixed number of subsets. If you are familiar with algebra, it helps to think about the distribution of degrees in the monomials of multivariate polynomials. If you are familiar with calculus, think about the derived terms in the multivariate Taylor expansion.

From these analogies, it should be clear that the number of ways to obtain $j$ realisations of a particular event amongst $N$ trials is $\binom{N}{j}$. Going back to probabilities, in order for this particular outcome to happen, we need some event $k$ to happen exactly $j$ times, for which the probability is $p_k^j (1-p_k)^{N-j}$.

Hence the formula that @rodms gave you. But what you are asking goes one step further than that. There are many ways that $Z_j$ can be equal to any number $m$. To clarify: $\mathrm{Pr}(\ Z_j = m\ )$ is the probability that exactly $m$ events occur $j$ times.

In order to obtain this formula, there is a second combinatorial layer; namely the combinations of $m$ events that would occur $j$ time, of which there are exactly $\binom{K}{m}$. Therefore, $\mathrm{Pr}(\ Z_j = m\ )$ is given by the sum over these combinations that the selected $m$ events jointly occur $j$ time.

For any such combination of $m$ events with indices $i_1\neq ...\neq i_m$, the probability that $(X_{i_1} = j\ \text{ and }\ X_{i_2} = j\ ...\ \text{ and }\ X_{i_m} = j)$ if $mj\leq N$ is given by: $$ \binom{N}{mj} \pi_{(i_1, ... , i_m)}^j (1- \sigma_{(i_1, ... , i_m)})^{N-mj} $$ where $\pi_{(i_1, ... , i_m)} = \prod_{k=1}^m p_{i_k}$ and $\sigma_{(i_1, ... , i_m)} = \sum_{k=1}^m p_{i_k}$. This yields the (rather ugly) formula: $$ \mathrm{Pr}(\ Z_j = m\ ) = \sum_{i_1\neq ...\neq i_m} \binom{N}{mj} \pi_{(i_1, ... , i_m)}^j (1- \sigma_{(i_1, ... , i_m)})^{N-mj} $$ where this sums iterates over the $\binom{K}{m}$ combinations mentioned previously.

If we further assume all $p_i$ equal, then this simplifies to: $$ \mathrm{Pr}(\ Z_j = m\ ) = \binom{K}{m} \binom{N}{mj} p^{mj} (1- mp)^{N-mj} $$