ُIs the exponential term necessary in logsoftmax when having positive inputs?

70 Views Asked by At

Considering the logsoftmax function $f:\mathbb{R}^n\rightarrow\mathbb{R}^n$ as: $$[f(x)]_i =-\log\left(\frac{e^{x_i}}{\sum_j{e^{x_j}}}\right)\quad i=1,2,\dots,n$$ where $x > 0 $ and is the decision vector which needs to be optimized to have more confidence in $i$-th decision as the $i$-th element of $x$. So one can add/use $f(x,i)$ in an optimization setting. Also assume $i$ is fixed.

But also if we define $g:\mathbb{R}^n\rightarrow\mathbb{R}^n$: $$[g(x)]_i =-\log\left(\frac{{x_i}}{\sum_j{{x_j}}}\right) \quad i=1,2,\dots, n$$ it has its minimum where $x_i$ is higher than other elements in $x$.

So considering we maintain the solution set as $x > 0$, then what would be the benefit of using $f(x,i)$ over $g(x,i)$?

Does it provide better convexity terms in an optimization setting or any other relevant benefit?