I recently came about the following thought experiment about compounding Bernoulli distribution.
At level 0, it is a simple Bernoulli distribution with success probability $p$: $$X= \begin{cases} 1 & \text{w/p} ~p\\ 0 & \text{w/p} ~1-p. \end{cases} $$
At level 1, we found $X$ is actually a compound distribution. $X$ is $X_1$ with probability $p_1$ and $X_0$ with probability $p_0=1-p_1$. On this higher level, we have
$$X_1= \begin{cases} 1 & \text{w/p} ~q_{1}\\ 0 & \text{w/p} ~1-q_{1} \end{cases} $$
$$X_0= \begin{cases} 1 & \text{w/p} ~q_{0}\\ 0 & \text{w/p} ~1-q_{0} \end{cases} $$ For consistency, we should have $p=q_{1}p_1+q_{0}p_0$.
At level 2, we can again define $$X_1= \begin{cases} X_{11} & \text{w/p} ~p_{11}\\ X_{10} & \text{w/p} ~1-p_{11} \end{cases} $$ and $$X_0= \begin{cases} X_{01} & \text{w/p} ~p_{01}\\ X_{00} & \text{w/p} ~1-p_{01} \end{cases} $$ where $X_{ij}$ takes value $1$ w/p $q_{ij}$. Similarly, for consistency, we should have $$p=p_{11}q_{11}+p_{10}q_{10}+p_{01}q_{01}+p_{00}q_{00}$$ and $$p_{11}+p_{10}+p_{01}+p_{00}=1.$$
If we keep doing this iteratively, at level $k$, we should have $2^{k}$ Bernoulli random variables (e.g., $X_{1,\cdots,1})$ with $2^k$ weights (e.g., $p_{1,\cdots,1})$, which together form a compound distribution of a categorical distribution and Bernoulli distributions.
My first question is that if we keep doing this indefinitely, what will the distribution look like? Will it converge to a continuous distribution of some known kind?
Secondly, if such a final form of distribution exists, let's say it is a random variable $Y$. If we run one more level of compounding, what will happen? Should we have $Y=p'Y_1+(1-p')Y_0$ for some non-identical $Y_1$ and $Y_0$? or should we have to have $Y=Y_1=Y_0$ as $Y$ is the final form?