Let $u,v:\mathbb R \to \mathbb R$ and $\phi: \mathbb R \to \mathbb R_+$ be smooth bounded functions. Assume also $\phi'\le 0$. Assume that $u(0) - v(0) = 0$ and that $0$ is a strict global minimum of $u-v$. Let us assume $D_\epsilon = \{x: u(x)-v(x) < \epsilon\}\subset B_1(0)$. Under these assumptions, is it possible to determine the sign of $$\int_{D_\epsilon} \phi(x) \left(\int_{\mathbb R} \frac{u(x+z) -v(x+z) -v(x)-u(x)}{|z|^{1+\alpha}} dz\right) dx $$ for some $\alpha \in (0,2)$? (positive or negative?)
Note that, if we had $$\int_{D_\epsilon} \phi(x)\partial_x(u-v) dx$$ instead of the previous expression with "fractional derivative", I would compute $$\int_{D_\epsilon} \phi(x)\partial_x(u-v)dx = \int_{D_\epsilon} \phi(x)\partial_x(\min\{u-v-\epsilon,0\}) dx = \int_{B_1} \phi(x)\partial_x(\min\{u-v-\epsilon,0\}) dx\ge 0$$ (becasue the function is continuous and identically zero on a neighborhood of $\partial B_1$).
Yes you can if you are allowed to choose $\varepsilon$. It also doesn't require the assumption $\phi'\leqslant 0$ only that $\phi>0$. Let $w=u-v$. Then (up to a positive normalisation constant), $$\int_{\mathbb R} \frac{u(x+z)-v(x+z)-u(x)-v(x)}{\vert z \vert^{n+\alpha}} dz = \int_{\mathbb R} \frac{w(x+z)-w(x)}{\vert z \vert^{n+\alpha}} dz =-(-\Delta)^{\frac\alpha2}w(x)$$ where $(-\Delta)^{\frac\alpha2}$ is the fractional Laplacian. Since $w(0)=$ and $w(x)>0$ provided $x \neq 0$, we have $$ (-\Delta)^{\frac\alpha2}w(0) =-\int_{\mathbb R} \frac{w(x+z)}{\vert z \vert^{n+\alpha}} dz <0.$$ Since $w$ is smooth we also have $(-\Delta)^{\frac\alpha2}w$ is smooth hence there exists $\delta >0$ such that $(-\Delta)^{\frac\alpha2}w <0$ on $ (-\delta,\delta).$ We may choose $\varepsilon$ sufficiently small such that $D_\varepsilon \subset (-\delta,\delta)$. Then $$ \int_{D_\varepsilon}\phi (x) \int_{\mathbb R} \frac{u(x+z)-v(x+z)-u(x)-v(x)}{\vert z \vert^{n+\alpha}} dz\, dx =-\int_{D_\varepsilon}\phi (x) (-\Delta)^{\frac\alpha2}w(x)\, dx >0.$$