The exponential smoothing function is $f:\Bbb R^n\to \Bbb R$ defined as
$$f(x):= \log\left(\sum_{i=1}^{n}e^{x_i}\right).$$ Obviously, when $x_i$ is large for some $i$, the term inside the logarithmic function goes to infinity. How should we implement the exponential smoothing function in a computer so as to avoid overflow?
The only thing I can think of is to use the identity
$$f(x):= \log\left(\sum_{i=1}^{n}e^{(x_i- L)}\right) + L$$ for any $L$, and choose $L$ large enough. Are there other practical(better) ways to do this?
Let $L=x_n=\max_ix_i$ wlog. The function may then be written as
$$f=x_n+\log\left(1+\sum_{i=1}^{n-1}\exp(x_i-x_n)\right)$$
which does not incur any overflow for any individual terms, which are all less than $1$. It is worth noting here that for small $x$ one has
$$\log(1+x)\sim x$$
so that if the exponential terms in the sum underflow it should not be an issue, since
$$x_n+\log(1+\epsilon)\sim x_n+\epsilon\sim x_n$$
cannot be differentiated in floating point arithmetic unless $x_n=0$.
Aside from over/underflow, this form has the advantage of one less exponential term to compute and it is more accurate to compute $\operatorname{Lnp1}(x)=\log(1+x)$ directly for small $x$ instead of adding $1$ and then taking the logarithm. See here.