Given $Z\sim$ Gaussian Mixture model: $$p(z) = \sum_{i=1}^n \pi_i \cdot g_i(z) $$ with $g_i(z) = \mathcal{N}(z;\mu_i,\sigma^2_i),$ define $\phi = \{\mu_i,\sigma^2_i,\pi_i\}_{i=1}^n.$
I want to find a low variance estimate for the entropy $\nabla_\phi \mathcal{H}(q(z)).$ At least for the gradients with respect to $\mu_i,\sigma^2_i$ I was thinking of writing:
$$ \nabla \mathcal{H}(q(z)) = \sum_{i=1}^n \pi_i \cdot \nabla \mathbb{E_{Z\sim g_i(\cdot)}[\log p(z)]} $$ together with the $\epsilon$ reparametrization trick for each component $g_i(\cdot).$
Is there a better way of doing it, i.e. estimator with lower variance?
What would be the best way to compute derivatives with respect to the mixture weights? We could just have the same approach as above. In this case, would we know if this estimator has high variance?
How could I test this empirically?
Thanks!!!