By the orbit-stabilizer theorem since the action is transitive then an orbit $\{gPg^{-1}: g\in G\} =n_p$ is equal to the number of $p$ sylow subgroups in a group $|G|=p^{\alpha}s$ with $(p^\alpha,s)=1$ and we get that $$|G/Stab_G(P)|=|G/\{g\in G: gPg^{-1}=P\}|=\frac{p^\alpha s}{|N_G(P)|}=n_p \text{ with } \begin{cases} n_p\equiv 1\mod{p} \\ n_p\mid s \end{cases}$$
The second condition on $n_p$ implies that $|N_G(P)|=p^\beta$ for some $\beta\le\alpha$
The first condition seems imply that $|N_G(P)|=p^\alpha\implies\beta=\alpha$
I'm not sure where I'm going with this but I'd like to know what characterizes a group that acts transitively/not transitively...
The second condition implies that $\lvert N_G (P) \rvert = p^\alpha\left(\dfrac{s}{n_p}\right)$. This does not imply that $\lvert N_G (P) \rvert$ is a power of the prime $p$. What it does imply, however (since $\gcd (p^\alpha, s) = 1$), is that the highest power of $p$ that divides the integer $\lvert N_G (P) \rvert$ is $p^\alpha$.
P.S. The correct relationship between a $p$-Sylow subgroup $P$ of $G$ and its normaliser is that $[G:N_G (P)] = n_p$. There is a bijection between all conjugates of $P$ (i.e. all the $p$-Sylow subgroups of $G$) and the left cosets of $N_G (P)$. The bijection is $gPg^{-1} \mapsto gN_G (P)$.