If $x\sim\mathcal{N}(\mu, \sigma^2)$, what is $\mathbb{E}\left(\frac{\exp(x)}{a+\exp(x)}\right)$ for $a>0$?

Question

Let $x\sim\mathcal{N}(\mu, \sigma^2)$ be a normal random variable. I'm interested in a new random variable, defined as $$ z = \frac{\exp(x)}{a+\exp(x)},\quad x\in\mathbb{R}, a>0. $$ More specifically, I would like to find the closed form (if any) of its expected value, $\mathbb{E}[z]$, but I don't know how to proceed.

As a first step I could use an intermediate random variable $y=\exp(x)$, which follows the log-normal distribution (since $x$ is normal) and its mean and variance are given w.r.t. to $x$'s mean and variance. But again, I'm not sure if that helps.

Answer 1

$F_Z(z)=P(Z\le z)=P(\frac{1}{1+ae^{-X}}\le z) = P(Ln(\frac{az}{1-z}) \le X) = F_X(Ln(\frac{az}{1-z}) )$

So $f_Z(z)=\frac{\partial}{\partial z}F_X(Ln(\frac{az}{1-z}) ) = \frac{1}{z(1-z)} f_X(Ln(\frac{az}{1-z}))$. Where $f_X$ is the usual normal density function. I don't think this will leave you much wiser so better to go after $E[Z]$ directly.

Answer 2

You want to compute the expectation of $\frac{\exp(X)}{a+\exp(X)}$ for $X \gets \mathcal{N}(\mu,\sigma^2)$ with $a>0$.

Firstly, we can simplify the problem by removing one of the parameters. We have $\frac{\exp(X)}{a+\exp(X)} = \frac{1}{1+a\cdot\exp(-X)} = \frac{1}{1+\exp(-(X-\log a))}$. Thus $\underset{X \gets \mathcal{N}(\mu,\sigma^2)}{\mathbb{E}}\left[\frac{\exp(X)}{a+\exp(X)}\right] = \underset{X \gets \mathcal{N}(\mu-\log a,\sigma^2)}{\mathbb{E}}\left[\frac{1}{1+\exp(-X)}\right].$ So without loss of generality $a=1$.

The function $f(x) = \frac{1}{1+\exp(-x)}$ is known as the logistic function.

Thus the distribution of $f(X)$ for $X \gets \mathcal{N}(\mu,\sigma^2)$ is the logit-Normal distribution. Unfortunately, per Wikipedia the mean has no analytic expression.

If you want to compute the mean nonetheless, this article may be useful:

John B. Holmes & Matthew R. Schofield (2022) Moments of the logit-normal distribution, Communications in Statistics - Theory and Methods, 51:3, 610-623, DOI: 10.1080/03610926.2020.1752723

In particular, the above referenced paper gives the following formula for the mean:

---

Now an informal explanation for why this is so hard to compute: We could compute $\mathbb{E}[f(X)]$ by computing a Taylor series $f(x) = \sum_{k=0}^\infty \frac{f^{(k)}(\mu)}{k!} (x-\mu)^k$ and then using the moments of the Gaussian, i.e., $$\mathbb{E}[f(X)] = \sum_{k=0}^\infty \frac{f^{(k)}(\mu)}{k!} \mathbb{E}[(X-\mu)^k] = \sum_{\ell=0}^\infty \frac{f^{(2\ell)}(\mu)}{(2\ell-1)!!}.$$ (Since $f(x) = \frac12 + \frac12 \tanh\left(\frac{x}{2}\right)$, we could use the Taylor series for $\tanh$.) Unfortunately, $f$ has a pole at the value $\pi i$ in the complex plane, as $1+\exp(\pi i) = 0$. Even though we aren't interested in complex values, this tells us that the Taylor series has a finite radius of convergence. Since we are evaluating this on a Gaussian which is supported on the entire real line, this is a problem.