Let $n$ and $b$ be integers larger than $1$, and write the base-$b$ expansion of $n$ as $n = \sum\limits_{i=0}^\infty a_i b^i$ (only finitely many of the $a_i$ are nonzero). We can define the polynomial $P_b^n(X)$ associated to this expansion by $P_b^n(X) = \sum\limits_{i=0}^\infty a_iX^i$.
Now define the set $S_n = \{b \mid b \lt n \text{ and } P_b^n(X)\text{ is reducible in }\mathbb Z[X]\}$. Note that a linear polynomial $a_0+a_1X$ can also be considered reducible if $\gcd(a_0, a_1) \gt 1$.
What do we know about lower/upper bounds and asymptotic behavior for $|S_n|$ (particularly when $n$ is composite)?
The motivation for this question is that given an integer $n$, we can try to factor it by finding a base $b$ in which $P_b^n(X)$ is reducible, and obtaining factors of $n$ by replacing $X$ with $b$ in the polynomial factors. I am interested in the likelihood that such a strategy may succeed. Of course I am aware of better integer factorization algorithms, such as the Quadratic Sieve or the General Number Field Sieve, but I want to know how such an algorithm would perform, and whether/how often it can fail on composite numbers.