Statement of the Problem
We wish to show that the following norm:
$ \large || \int^{t/2}_{0} \xi_1 e^{-(t-s)|\xi|^{\alpha}} \int_{\eta \in \mathbb{R}^2} \frac{|\xi|^2 \eta_2 e^{-(S+1)|\xi - \eta|^{\alpha}} e^{-(s+1)|\eta|^{\alpha}}}{ |\eta| |\eta - \xi| (|\eta| + |\eta - \xi|) } \ \text{d}\eta \ \text{d}s ||_{L^2(|\xi| \leq \epsilon t^{-1/\alpha})}$,
has "lower order" than $O(\epsilon^3t^{1-\frac{4}{\alpha}})$, where $\alpha \in (1,2)$. Here, $\epsilon \in \mathbb{R}$ is a constant taken smaller than $1$, and $t \in (0, \infty)$ represents time in a partial differential equation that this problem is related to.
That is, we wish to show either
$ \large || \int^{t/2}_{0} \xi_1 e^{-(t-s)|\xi|^{\alpha}} \int_{\eta \in \mathbb{R}^2} \frac{|\xi|^2 \eta_2 e^{-(S+1)|\xi - \eta|^{\alpha}} e^{-(s+1)|\eta|^{\alpha}}}{ |\eta| |\eta - \xi| (|\eta| + |\eta - \xi|) } \ \text{d}\eta \ \text{d}s ||_{L^2(|\xi| \leq \epsilon t^{-1/\alpha})} \leq C_\alpha \epsilon^{3+\delta}t^{1-\frac{4}{\alpha}} $
or $\leq C_\alpha t^{1-\frac{4}{\alpha} - \delta}$, (power of $\epsilon$ won't matter if we can achieve a lower power of $t$!)
for some $\delta >0$.
Progress So Far
We divide the integral over $\mathbb{R}^2$ into three parts: one with $|\eta| \leq \frac{1}{2} |\xi|$, one with $|\eta| \in (\frac{1}{2}|\xi|, 2|\xi|)$, and lastly one with $|\eta| \geq 2|\xi|$.
We can use basic inequalities to obtain the desired result for the integral over small $\eta$:
$ \large \int_{|\eta| \leq \frac{1}{2}|\xi| } \frac{|\xi|^2 \eta_2 e^{-(S+1)|\xi - \eta|^{\alpha}} e^{-(s+1)|\eta|^{\alpha}}}{ |\eta| |\eta - \xi| (|\eta| + |\eta - \xi|) } \ \text{d}\eta \leq C_\alpha \int_{|\eta| \leq \frac{1}{2}|\xi| } \frac{|\xi|^2 |\eta| e^{-(s+1)|\eta|^{\alpha}}}{ |\eta| |\xi|^2 } \ \text{d}\eta $
(here we have bounded one of the gaussians by $1$, and the constant $C_\alpha$ emerges from bounding the denominator)
$ \large = C_\alpha \int_{|\eta| \leq \frac{1}{2}|\xi| } e^{-(s+1)|\eta|^{\alpha}} \ \text{d}\eta \leq C_\alpha \int_{|\eta| \leq \frac{1}{2}|\xi| } 1 \ \text{d}\eta = C_\alpha |\xi|^2 $
Thus the overall norm, taking only the integral for small $\eta$, is bounded as follows:
$\large || \int^{t/2}_{0} \xi_1 e^{-(t-s)|\xi|^{\alpha}} \int_{|\eta| \leq \frac{1}{2}|\xi|} \frac{|\xi|^2 \eta_2 e^{-(S+1)|\xi - \eta|^{\alpha}} e^{-(s+1)|\eta|^{\alpha}}}{ |\eta| |\eta - \xi| (|\eta| + |\eta - \xi|) } \ \text{d}\eta \ \text{d}s ||_{L^2(|\xi| \leq \epsilon t^{-1/\alpha})} $
$ \large \leq || \int^{t/2}_{0} \xi_1 e^{-(t-s)|\xi|^{\alpha}} C_\alpha |\xi|^2 \ \text{d}s ||_{L^2(|\xi| \leq \epsilon t^{-1/\alpha})} \leq C_\alpha || |\xi|^3 \int^{t/2}_{0} 1 \ \text{d}s ||_{L^2(|\xi| \leq \epsilon t^{-1/\alpha})} $
$ \large = C_\alpha \epsilon^4 t^{1-\frac{4}{\alpha}} $, which is what we want!
Calculation is similar for $|\eta| \in (\frac{1}{2}|\xi|, 2|\xi|)$.
Remaining Problem
The work remains to be done for the integral over $\eta$ large. The trick in these calculations seems to be in keeping $|\xi|$ to a power higher than $2$, thus giving us a higher power of $\epsilon$, and otherwise approximating everything else by a constant. This is proving hard to do for the $\eta$ large integral, however, since we cannot simply estimate the gaussian function by a consant anymore.
One other method might be to keep only $|\xi|^2$, but achieve a power of $s$ lower than $-\frac{1}{\alpha}$, which would give us the same power of $\epsilon$, but a lower power of $t$ in the end.
My Attempts
So far, I have tried to get a similar result using the same kind of inequalities as above. It is easy to bound the norm by $C_\alpha \epsilon^3 t^{1-\frac{4}{\alpha}}$, but unfortunately difficult to increase the power of $\epsilon$ or decrease the power of $t$. Here is how I showed this bound:
$ \large \int_{|\eta| \geq 2|\xi| } \frac{|\xi|^2 \eta_2 e^{-(S+1)|\xi - \eta|^{\alpha}} e^{-(s+1)|\eta|^{\alpha}}}{ |\eta| |\eta - \xi| (|\eta| + |\eta - \xi|) } \ \text{d}\eta \leq C_\alpha \int_{|\eta| \geq 2|\xi| } \frac{|\xi|^2 |\eta| e^{-(s+1)|\eta|^{\alpha}}}{ |\eta|^2 |\xi| } \ \text{d}\eta $
Note that we have had to estimate the denominator here differently from before. This is because the previous strategy of estimating the gaussian by a constant before integrating of course fails here.
$ \large = C_\alpha |\xi| \int_{|\eta| \geq 2|\xi| } \frac{e^{-(s+1)|\eta|^{\alpha}}}{ |\eta|} \ \text{d}\eta \leq C_\alpha |\xi| \int_{\eta \in \mathbb{R}^2} \frac{e^{-(s+1)|\eta|^{\alpha}}}{ |\eta|} \ \text{d}\eta \leq C_\alpha |\xi| (s+1)^{-\frac{1}{\alpha}} $
Which gives us the following estimate for the norm:
$\large || \int^{t/2}_{0} \xi_1 e^{-(t-s)|\xi|^{\alpha}} \int_{|\eta| \geq 2|\xi|} \frac{|\xi|^2 \eta_2 e^{-(S+1)|\xi - \eta|^{\alpha}} e^{-(s+1)|\eta|^{\alpha}}}{ |\eta| |\eta - \xi| (|\eta| + |\eta - \xi|) } \ \text{d}\eta \ \text{d}s ||_{L^2(|\xi| \leq \epsilon t^{-1/\alpha})} $
$ \large \leq || \int^{t/2}_{0} \xi_1 e^{-(t-s)|\xi|^{\alpha}} C_\alpha |\xi| (s+1) \ \text{d}s ||_{L^2(|\xi| \leq \epsilon t^{-1/\alpha})} \leq C_\alpha || |\xi|^2 ((\frac{t}{2} +1)^{1-\frac{1}{\alpha}} -1) ||_{L^2(|\xi| \leq \epsilon t^{-1/\alpha})} $
$ \large \leq C_\alpha || |\xi|^2 (\frac{t}{2} +1)^{1-\frac{1}{\alpha}} ||_{L^2(|\xi| \leq \epsilon t^{-1/\alpha})} + C_\alpha || |\xi|^2||_{L^2(|\xi| \leq \epsilon t^{-1/\alpha})} $
$ \large = O(\epsilon^3t^{1-\frac{4}{\alpha}}) $.
The difficulty lies in getting this inequality just one bit smaller wrt $t$ or $\epsilon$.
It turns out the solution is very simple. Instead of focusing only on whole powers of $|\eta|, \ |\xi|$, we can just use a fractional power of each in one of our inequalities to get the desired result. That is:
$ \large \int_{|\eta| \geq 2|\xi| } \frac{|\xi|^2 \eta_2 e^{-(S+1)|\xi - \eta|^{\alpha}} e^{-(s+1)|\eta|^{\alpha}}}{ |\eta| |\eta - \xi| (|\eta| + |\eta - \xi|) } \ \text{d}\eta \leq C_\alpha \int_{|\eta| \geq 2|\xi| } \frac{|\xi|^2 |\eta| e^{-(s+1)|\eta|^{\alpha}}}{ |\eta|^{5/2} |\xi|^{1/2} } \ \text{d}\eta $
$ \large = C_\alpha |\xi|^{3/2} \int_{|\eta| \geq 2|\xi| } \frac{ e^{-(s+1)|\eta|^{\alpha}}}{ |\eta|^{3/2} } \ \text{d}\eta \leq C_\alpha |\xi|^{3/2} (s+1)^{-1/2\alpha} \int_{0}^{\infty} \frac{ e^{-r^{\alpha}}}{ r^{1/2} } \ \text{d}r $
$ \large = C_\alpha |\xi|^{3/2} (s+1)^{-1/2\alpha}$
This will overall give us terms of orders $\epsilon^{4} t^{1-\frac{4}{\alpha}}$, and $ \epsilon^{3.5} t^{1-\frac{4}{\alpha}} $ in our estimate of the entire norm written at the top of the question, which is just barely safe; so we have achieved the desired result of showing that the overall norm grows more slowly than $O(\epsilon^3 t^{1-\frac{4}{\alpha}}) $.