Product of Several Functions Becomes Very Small: Scaling?

38 Views Asked by At

I have the following ratio:

$$\frac{\sum_{i = 1}^n Y_i \prod_{p = 1}^P \lambda_p^{z_{i,p}}}{\sum_{i = 1}^n \prod_{p = 1}^P \lambda_p^{z_{i,p}}}$$

where $\lambda_p \in (0,1]$ is a parameter, and $z_{i,p} \in \{0,1\}$ is an indicator variable.

I believe that evaluating this ratio for large $P$ (e.g., $P = 50$) within a routine is causing numerical instability. For example, the numerator and denominator may be on the order of $10^{-100}$. You can think of the product terms as kernel products to approximate multivariate densities.

There are a few ways to rewrite the product terms, but the problem persists. I would appreciate if you have any ideas to "scale" the numerator and denominator, such that they take on values close to $1$ and not $10^{-100}$.

EDIT:

As was suggested, I could try a log transformation of the products. The approach I've been using is the following. Define

$$q_i = \prod_{p = 1}^P \lambda_p^{z_{i,p}}$$

Thus,

$$ \log q_i = \sum_{p = 1}^n z_{i,p}\log \lambda_p$$

And the original ratio becomes:

$$\frac{\sum_{i = 1}^n Y_i \exp \left(\log q_i\right)}{\sum_{i = 1}^n \exp \left(\log q_i\right)}$$

Now, I have sums of exponentials that take on very small values, and the problem persists.