Identifiability of a probability distribution

162 Views Asked by At

Is the following distribution

$$P(\textbf{x}; \boldsymbol{\mu}, \boldsymbol{\Sigma}) = \int_{\text{h} \in \mathbb{R}^d} N\left(\textbf{h}; \boldsymbol{\mu}, \boldsymbol{\Sigma}\right) \,\text{Mult}\left(\textbf{x}; \pi(\textbf{h})\right) \text{d}\textbf{h}$$

where $\pi(\textbf{h}) = \frac{\left(1, e^{h_1}, \dots, e^{h_d}\right)}{1+e^{h_1}+\dots+e^{h_d}}$, $N\left(\textbf{h}; \boldsymbol{\mu}, \boldsymbol{\Sigma}\right)$ the multivariate normal distribution and $\text{Mult}\left(\textbf{x}; \pi\right)$ is the multinomial distribution, identifiable?

I.e. $$P(\textbf{x}; \boldsymbol{\mu}_1, \boldsymbol{\Sigma}_1) = P(\textbf{x}; \boldsymbol{\mu}_2, \boldsymbol{\Sigma}_2) \implies \boldsymbol{\mu_1} = \boldsymbol{\mu_2} \text{ and } \boldsymbol{\Sigma_1} = \boldsymbol{\Sigma_2}$$

I've found different sources dealing with a similar problem but I fail to find a final answer.

  • Lindsay, B. G. (1995). Mixture Models: Theory, Geometry and Applications. Institute of Mathematical Statistics (Vol. 5).

  • Chandra, S. (1977). On the mixtures of probability distributions. Scandinavian Journal of Statistics, 4, 105–112.

  • Hasselblad, V. (1969). Estimation of Finite Mixtures of Distributions from the Exponential Family. Journal of the American Statistical Association, 64(328), 1459–1471.

1

There are 1 best solutions below

4
On BEST ANSWER

To know the values of $P(\mathbf x;\mathbf \mu,\mathbf \Sigma)$ for all $\mathbf x$ is to know all the mixed moments of the random vector $\pi(\mathbf h)$ and hence (because the range of $\pi$ is bounded) the distribution of the random vector $\pi(\mathbf h)$.

This follows from the algebraic form of the multinomial probability function: $P(\mathbf x|\mathbf \pi)= m(\mathbf x) \mathbf \pi^{\mathbf x}$ where $m(\mathbf x)$ denotes the multinomial coefficient for count vector $\mathbf x$ and ${\mathbf \pi}^{\mathbf x} =\prod \pi_i^{x_i}$. So the compound multinomial probability function $P(\mathbf x)=E[ P(\mathbf x|\mathbf \pi)] $ that you get when you take the expectation over $\mathbf \pi$ is thus equal to $m(\mathbf x)$ times the $\mathbf x$-th moment of the random vector $\mathbf \pi$. (The $\mathbf x$-th moment with respect to the mixing distribution for $\mathbf \pi$.) Finally, its a technical fact that the moments of a random vector distributed over a compact set, like the $d+1$ probability simplex, as your $\pi(\mathbf h)$ is, uniquely specifies the the distribution of that random vector.

So the question reduces to asking if the distribution of $\pi(\mathbf h)$ is identifiable.

Which it is: the map $\mathbf h\mapsto\pi(\mathbf h)$ is invertible, so to know the distribution of $\pi(\mathbf h)$ is to know the distribution of $\mathbf h$.

Why is $\pi(\mathbf h)$ invertible? We can recover $\mathbf x=(x_0,x_1,\ldots,x_d)$ by $h_i = \log(\pi_i(\mathbf h)/\pi_0(\mathbf h))$, when we number the subscripts $0$ through $d$.