Stratified Sampling $E(E(X \mid \mu, \sigma))$

47 Views Asked by At

Let $X$, $\mu$ and $\sigma$ be random variables. I want to estimate $E(X)$ using Monte Carlo. I am able to sample from, and know in closed-form, both the conditional distribution of $X \mid (\mu, \sigma)$ and the joint distribution of $(\mu, \sigma)$, but not the marginal distribution of $X$. To use the information I have, I write $$ E(X) = E(E(X \mid (\mu, \sigma))). $$ A crude estimator would be $$ \frac{1}{N}\sum_{i=1}^N \frac{1}{n}\sum_{j=1}^{n}X_{ij}, $$ where $X_{ij}$ is the $j^{th}$ observation in the sample from the conditional distribution $X \mid \mu_i, \sigma_i$. This requires $N$ samples, each of size $n$.

I understand that stratified sampling is a good candidate for variance reduction in such a scenario, but I'm not sure how to use it in this case. For stratified sampling we partition the range of $(\mu, \sigma)$ into $K$ strata $A_1, \ldots, A_K$ such that $\sum_{i=1}^K P((\mu, \sigma) \in A_i) = 1$. Letting $p_i = P((\mu, \sigma) \in A_i)$ we then write $$ E(X) = \sum_{i=1}^k p_i \,E(X \mid (\mu, \sigma) \in A_i), $$ and draw samples from $X \mid (\mu, \sigma) \in A_i$ to estimate the expectations. I know $p_i$ since I know the joint distribution of $(\mu, \sigma)$, but I can't figure out how to sample from $X \mid (\mu, \sigma) \in A_i$; rather, only $X \mid (\mu, \sigma)$, meaning only when $\mu$ and $\sigma$ are given values.

The example I have in mind is $X \mid \mu, \sigma \sim \mathcal{N}(\mu, \sigma^2)$, with some continuous uniform distribution on $(\mu, \sigma)$. How can I generate samples from $X \mid (\mu, \sigma) \in A_i$?