I am faced with the following problem:
Problem. $X$ and $Y$ are independent normal random variables both with zero mean, $\operatorname{Var}(X)=1,\operatorname{Var}(Y)<1$ (seems not crucial). Find $f$ maximizing $$E[(f(X)-f(X+Y))X], $$ where $f$ is nondecreasing, odd and smooth, satisfying $|f|\leq 1$.
The restriction "smooth" is added by me to make the problem seem simpler. It can also be replaced by other restrictions. If possible, can we find the target $f$ when it is only subject to nondecreasing, odd and bounded?
I would be grateful if there is any hint because I am totally not familliar with finding function to optimize some target.
Let $I(f)$ denote the expectation, and let $\sigma^2 = \mathbf{Var}(Y)$. Also, write $X' = X + Y$. Using the identity
$$ \mathbf{E}[X \mid X'] = \frac{1}{1+\sigma^2} X', $$
$I(f)$ can be recast as
\begin{align*} I(f) &= \mathbf{E}[X f(X)] - \frac{1}{1+\sigma^2}\mathbf{E}[X'f(X')] \\ &= 2 \left( \mathbf{E}[X f(X); X \geq 0] - \frac{1}{1+\sigma^2}\mathbf{E}[X'f(X'); X' \geq 0] \right) \end{align*}
Then by invoking the identity $f(x) = \int_{0}^{\infty} f'(t) \mathbf{1}_{\{t \leq x\}} \, \mathrm{d}t$ for $x \geq 0$,
\begin{align*} I(f) &= 2 \int_{0}^{\infty} f'(t) \left( \mathbf{E}[X \mathbf{1}_{\{t \leq X\}}] - \frac{1}{1+\sigma^2} \mathbf{E}[X' \mathbf{1}_{\{t \leq X'\}}] \right) \, \mathrm{d}t \\ &= \frac{2}{\sqrt{2\pi}} \int_{0}^{\infty} f'(t) \underbrace{\left( e^{-t^2/2} - \frac{1}{\sqrt{1+\sigma^2}} e^{-t^2/2(1+\sigma^2)} \right)}_{=:\varphi_{\sigma}(t)} \, \mathrm{d}t \end{align*}
Now, it is not hard to show that $ \varphi_{\sigma}(t) \leq \varphi_{\sigma}(0) = 1 - \frac{1}{\sqrt{1+\sigma^2}} $ for $t \geq 0$ with the equality precisely when $t = 0$. This and the fact that $f'(t) \geq 0$ together, we get
$$ I(f) < \frac{2}{\sqrt{2\pi}}\left(1 - \frac{1}{\sqrt{1+\sigma^2}}\right) =: I_{\max} $$
Moreover, for any $f$ satisfying the condition of the problem and for any $L > 0$,
\begin{align*} I(f(L\cdot)) &= \frac{2}{\sqrt{2\pi}} \int_{0}^{\infty} L f'(Lt) \varphi_{\sigma}(t) \, \mathrm{d}t \\ &= \frac{2}{\sqrt{2\pi}} \int_{0}^{\infty} f'(u) \varphi_{\sigma}(u/L) \, \mathrm{d}u \tag{$u=Lt$} \\ &\to \frac{2}{\sqrt{2\pi}} \int_{0}^{\infty} f'(u) \varphi_{\sigma}(0) \, \mathrm{d}u \tag{as $L\to\infty$} \\ &= I_{\max}. \end{align*}
Therefore, we conclude that $I_{\max}$ is the supremum (but not maximum) of the possible values of $I(f)$ for nondecreasing, odd, smooth $f$ with $|f| \leq 1$.
Remark. If we drop the smoothness condition for $f$, then $I(f)$ is maximized with the maximum value $I_{\max}$ when
$$ f(t) = \operatorname{sign}(t). $$