Just curious.
I had some values of $n$, and I wanted to conclude that all groups of order $n$ are solvable. Some were easy (eg. not divisible by 4, or $p^aq^b$). For some I did this by inspection of the list of small non-abelian simple groups and found that none of them had order dividing $n$.
In fact those $n$ I resorted to The List for, they had a prime factor $p$, and $n \lt \frac{p^3-p}{2} = \|PSL_2(p)\|$. It sure "looks" like in general we can say all groups of such orders are solvable. (EDIT: that is, if we already knew all groups of order $\frac{n}{p}$ were solvable, which in my cases I did. To rephrase: if we can say all groups of order $m$ are solvable, it "looks" like we can say all groups of order $mp$ are solvable, for any prime $p \gt \sqrt{2m+1}$. This would follow if $PSL_2(p)$ was the smallest unsolvable group with order divisible by $p$.)
I was curious if this was actually true.
For the weaker conjecture, YES. All groups order $m$ solvable implies all groups order $mp$ solvable, prime $p \gt \sqrt{2m+1}$.
Suppose all groups order $m$ are solvable, and there is an unsolvable group of order $mp$. If it has an unsolvable proper subquotient, then $p$ divides the order of this subquotient, else we would have an unsolvable group of order dividing $m$. So by induction we have a minimal simple group of order $m'p$, $m'$ dividing $m$.
These minimal simple groups have a short classification -- thank you j.p. -- they are (some of) the $PSL_2$'s, the Suzuki groups, and $PSL_3(3)$. The latter is ruled out by easy calculation.
The order of the Suzuki groups is $q^2(q^2+1)(q-1)$, for $q$ an odd power of 2. Fortunately $q^2+1 = (q+1+\sqrt{2q})(q+1-\sqrt{2q})$, and $\sqrt{2q}$ is an integer. So worst case we're going to have $p$ on the order of $\sqrt[4]{m}$, so there are only a few small cases to check. In fact the only exception is $q=2$, but ${}^2B_2(2)$ is solvable.
For $PSL_2(q)$ where $q$ a power of a prime but not itself prime, worst case that $q+1$ or $q-1$ equals prime $p$ or $2p$, but it is not hard to see still $p \le \sqrt{2m+1}$, equality only for $p=5$, and indeed $PSL_2(4) \simeq PSL_2(5)$.
So that leaves $PSL_2(p)$ for $p$ prime, for which $p = \sqrt{2m+1}$ exactly.
(EDITED) Not sure about the original question. If $PSL_2(p)$ is not always the smallest simple group whose order is divisible by $p$, it's not clear that this would contradict the above, after all, not all the $PSL_2(p)$'s are minimal. For instance for $p=19$ we have $m=180$ and it is not the case that all groups of order $m$ are solvable. Indeed $PSL_2(19)$ contains $PSL_2(5)$.