Given a vector in $\mathbb{R}^n$ I have an algorithm to compute
$$\sum_{i=1}^n x_i \log(x_i)$$
However for my application the norm of $x$ must be 1, hence for big $n$ the components tend to be too close to zero, so when I compute the log it often gets to NaN (in matlab language) instead of a number resulting in bugs...
I tried to compute the sum only for the $i$s when the $x_i$ is sufficiently different from $0$, but I still keep getting this kind of errors
Is there a stable manner to compute this, a way to regularize or something?
You just need that every $x_i$ is positive. By arranging them in increasing order $x_1\leq x_2\leq\ldots\leq x_n $ you can also exploit: $$\sum_{i=1}^{n}x_i\log x_i = (x_1+\ldots+x_n)\log x_n-\sum_{i=1}^{n-1}(x_1+\ldots+x_i)\log\frac{x_{i+1}}{x_i}$$ so you have to compute just one logarithm of a possibly big number and $n-1$ logarithms of numbers bigger than one but close to one, for which one of the two approximations: $$ \log\frac{x_{i+1}}{x_i} = -\log\frac{x_i}{x_{i+1}} \approx \sum_{n= 0}^N\frac{1}{n}\left(1-\frac{x_i}{x_{i+1}}\right)^n $$ $$ \log\frac{x_{i+1}}{x_i} \approx \sum_{n=0}^{N}\frac{(-1)^n}{n}\left(\frac{x_{i+1}}{x_i}-1\right)^n $$ is very effective depending on $\frac{x_{i+1}}{x_i}\geq 2$ or $\frac{x_{i+1}}{x_i}\leq 2$.