Given a fixed integer $a > 1$, what is the amount of integers $1 \leq n \leq x$ such that the multiplicative order of $a \pmod{n}$ is small?

Question

Let $x > 1$ be a large real variable and let $a > 1$ be a fixed integer. For an integer $n$ coprime to $a$, let $\text{ord}_n(a)$ denote the multiplicative order of $a$ modulo $n$. Fix an $0 < \epsilon < 1$ small. Can we give a good upper bound for the size of the following set? $$ \left\{1 \leq n \leq x \ : \ (n,a)=1 \text{ and } \text{ord}_n(a) \leq x^\epsilon\right\} $$