Exercise A-47 in Milne's Fields and Galois Theory notes asks to prove that if $p$ is a prime number and $F$ is a field of characteristic zero such that every irreducible polynomial $f(X)\in F[X]$ has degree $p^n$ for some non negative integer $n$, then every polynomial equation with coefficients in $F$ is solvable by radicals.
I already solved the exercise. The key is to show that the Galois group $\mathrm{Gal}(\overline{F}/F)$ is a $p$-group, where $\overline{F}$ is the algebraic closure of $F$.
Question: For a fixed prime $p$, is there any field $F$ satisfying the given hypothesis?
For example, for $p=2$ the answer is yes: any real closed field satisfies the hypothesis. What are such examples for other primes?
Hint: You can try to look at the union of the fields $\mathbb{F}_{l^n}$ inside $\overline{\mathbb{F}}_l$, for $n$ powers of $p$.
$\bf{Added:}$ for a more sophisticated example, see Iwasawa theory.
$\bf{Added:}$ Jyrki corrected me, so the above stuff has only documentary value. Now I have an idea, not sure how good it is:
Consider a field $F$ and its algebraic closure $\bar F$. The Galois group $G=\operatorname{Gal}(\bar F/F)$ is a profinite group. I think for these kind of groups we have Sylow subgroups. So let $P$ a $p$-Sylow subgroup of $G$. The subfield $K$ of $\bar F$ invariant by $P$ has absolute Galois group $\operatorname{Gal}(\bar F/K) = P$. It follows that every finite Galois extension of $K$ has degree a power of $p$.
Obs: in fact we don't need $P$ to be a Sylow pro-$p$ group, just simply a pro-$p$ group ( I vaguely understand this). But $K$ will corresponding to a Sylow will be the smallest inside $\bar F$ (up to conjugation).