I've been studying Sylow-$p$ subgroups, and I've come across this problem.
Let $G$ be a finite group. Show that the number of Sylow subgroups of $G$ is at most $\frac{2}{3}|G|$ . ($|G|$ is the number of elements of $G$).
I am having trouble figuring this one out, I was wondering if anyone could help?
On
Let $G$ be a group and let $$|G|=p_1^{a_1}p_2^{a_2}\cdots p_n^{a_n}$$ where $p_1<p_2<\cdots<p_n$ are distinct primes and $a_i\geq 1$ for all $i$. Let $N_p$ denote the number of Sylow $p$-subgroups and let $N$ denote the total number of Sylow subgroups, i.e. the sum of all $N_{p_i}$. By the fact that $N_p\equiv 1\pmod{p}$ and $N_p||G|$ we must have that $$N_{p_i}\leq \frac{|G|}{p_i^{a_i}}$$
Suppose $p_i\neq 2$ and $a_i=1$. Then $N_{p_i}$ is equal to the number of elements of order $p_i$ divided by $p_i-1$. Thus if $i_1<i_2<\cdots<i_m$ satisfy $a_{i_k}=1$ for all $k$ and these are all the indices for which the exponent is $1$, then $$\sum_{k=1}^{m}{N_{p_{i_k}}}\leq \frac{|G|}{p_{i_1}-1}$$
New, more elementary proof
There are five cases.
(1) $a_i\geq 2$ for all $i$.
In this case $$N\leq\sum_{p}{\frac{1}{p^2}|G|}$$ where $p$ ranges over all primes, and the sum of the squares of the recpirocals of all primes is less than or equal to $\frac{453}{1000}$, so $$N\leq \frac{453}{1000}|G|<\frac{2}{3}|G|$$
(2) $p_1=2$, $G$ has a normal $2$-complement, and $N_2<\frac{1}{2}|G|$
Let $H$ be the normal $2$-complement. Then $H$ contains all Sylow subgroups for primes other than $2$. Suppose $H$ is of order $2^{m-a_1}|G|$, where $m<a_1$. By induction on $n$ we have that $$\sum_{k>1}{N_{p_k}}\leq \frac{2}{3}|H|=\frac{2^{m+1}}{3\cdot 2^{a_1}}|G|\leq\frac{1}{3}|G|$$ Since $N_2<\frac{|G|}{2}$ we have that $N_2\leq \frac{|G|}{4}$, hence $$N\leq N_2+\frac{1}{3}|G|\leq \frac{1}{4}|G|+\frac{1}{3}|G|=\frac{7}{12}|G|<\frac{2}{3}|G|$$
(3) $p_1=3$ or $p_2=3$ and $G$ has a normal $3$-complement
This is similar to the previous case.
Let $H$ be the normal $3$-complement. Then $H$ contains all Sylow subgroups for primes other than $3$. Suppose $H$ is of order $3^{m-a_1}|G|$, where $m<a_1$. By induction on $n$ we have that $$\sum_{k\neq 2}{N_{p_k}}\leq \frac{2}{3}|H|=\frac{2\cdot 3^{m-a_1}}{3\cdot 3^{a_1}}|G|\leq\frac{2}{9}|G|$$ So $$N\leq N_3+\frac{2}{9}|G|\leq \frac{1}{3}|G|+\frac{2}{9}|G|=\frac{5}{9}|G|<\frac{2}{3}|G|$$
(4) $G$ has no normal $3$ complement, and if $p_1=2$ then $G$ also has no normal $2$-complement
The condition implies that $a_1\geq 2$, unless $p_1\geq 5$. In this case neither the Sylow $2$-subgroup nor the Sylow $3$-subgroup is in the center of its normalizer (unless each respective subgroup is trivial), so by Burnside's transfer theorem we have $N_2\leq\frac{1}{8}|G|$ (which also holds if there is no nontrivial Sylow $2$-subgroup) and $N_3\leq\frac{1}{6}|G|$ (which also holds if there is no nontrivial $3$-subgroup). The contribution of the $N_{p_i}$ for $p_i\geq 5$ such that $a_i=1$ is as usual at most $\frac{1}{4}|G|$. Thus $$N\leq\sum{\frac{1}{p^2}|G|}-\frac{1}{4}|G|-\frac{1}{9}|G|+\frac{1}{8}|G|+\frac{1}{6}|G|+\frac{1}{4}|G|\leq \frac{2851}{4500}|G|<\frac{2}{3}|G|$$
(5) $p_1=2$ and $N_2=\frac{1}{2}|G|$
In this case as below we are in the situation of Suppose that half of the elements of G have order 2 and the other half form a subgroup H of order n. Prove that H is an abelian subgroup of G., meaning that we have an abelian subgroup of index $2$, hence every Sylow $p_i$ subgroup for $i>1$ is normal, hence $$N=\frac{1}{2}|G|+n-1$$ and since in particular $n-1\leq\frac{|G|}{6}$ because $|G|\geq 2\cdot 3^{n-1}$ we have that $$N\leq \frac{2}{3}|G|$$ This bound is tight, with $S_3$ giving an example of equality (in fact $S_3$ is unique in this sense, which can be seen from the proof).
Old, overpowered proof (no need to read, preserved for posterity)
Our proof is broken into six cases (case 4 has some subcases though):
(1) $a_i\geq 2$ for all $i$
In this case $$N\leq\sum_{p}{\frac{1}{p^2}|G|}$$ where $p$ ranges over all primes, and the sum of the squares of the recpirocals of all primes is less than or equal to $\frac{23}{50}$, so $$N\leq \frac{23}{50}|G|<\frac{2}{3}|G|$$
(2) $a_i=1$ for some $i$ and either $p_1\neq 2$ or $p_1=2$ and $a_1\geq 4$
For $p$ prime we have $$\sum_{p>3}{\frac{1}{p^2}}\leq \frac{23}{50}-\frac{1}{4}-\frac{1}{9}=\frac{89}{900}$$ If $p_1\geq 3$ we have $$N\leq \sum_{k=1}^{m}{N_{p_{i_k}}}+\frac{89}{900}|G|\leq\frac{1}{p_1-1}|G|+\frac{89}{900}|G|\leq \frac{1}{2}|G|+\frac{89}{900}|G|=\frac{539}{900}|G|<\frac{2}{3}|G|$$ If $p_1=2$ we have assumed $a_1\geq 4$. If $p_2>3$, or $p_2=3$ and $a_2=1$ then since $$\frac{1}{16}+\sum_{p>3}{\frac{1}{p^2}}\leq \frac{23}{50}-\frac{1}{9}-\frac{3}{16}=\frac{581}{3600}$$ we have $$N\leq \sum_{k=1}^{m}{N_{p_{i_k}}}+\frac{581}{3600}|G|\leq \frac{1}{p_2-1}|G|+\frac{581}{3600}|G|\leq\frac{2381}{3600}|G|<\frac{2}{3}$$ If $p_2=3$ and $a_2>1$, let $p_k$ be the smallest integer such that $a_k=1$. Then since $$\frac{1}{16}+\sum_{p\geq 3}{\frac{1}{p^2}}\leq \frac{23}{50}-\frac{3}{16}=\frac{109}{400}$$ we have $$N\leq \sum_{k=1}^{m}{N_{p_{i_k}}}+\frac{109}{400}|G|\leq \frac{1}{p_k-1}|G|+\frac{109}{400}|G|\leq\frac{209}{400}|G|<\frac{2}{3}$$
(3) $p_1=2$, $G$ has a normal $2$-complement and $N_2<\frac{|G|}{2}$
Let $H$ be the normal $2$-complement. Then $H$ contains all Sylow subgroups for primes other than $2$. Suppose $H$ is of order $2^{m-a_1}|G|$, where $m<a_1$. By induction on $n$ we have that $$\sum_{k\neq 2}{N_{p_k}}\leq \frac{2}{3}|H|=\frac{2^{m+1}}{3\cdot 2^{a_1}}|G|\leq\frac{1}{3}|G|$$ Since $N_2<\frac{|G|}{2}$ we have that $N_2\leq \frac{|G|}{4}$, hence $$N\leq N_2+\frac{1}{3}|G|\leq \frac{1}{4}|G|+\frac{1}{3}|G|=\frac{7}{12}|G|<\frac{2}{3}|G|$$
(4) $p_1=2$, $a_1\leq 3$, the Sylow $2$-subgroups are abelian but not cyclic, and $G$ does not have a normal $2$-complement
The Walter theorem states that if $O(G)$ is the maximal normal subgroup of odd order, then $G/O(G)$ is a direct product of $2$-groups, projective special linear groups, Janko groups, and Ree groups of type ${}^2G_2$. Let $H$ be a Sylow $2$-subgroup. Then the number of distinct conjugates of $HO(G)$ is the same as the number of Sylow $2$-subgroups of $G/O(G)$. Call this number $N_2(G/O(G))$ and let $N_2(HO(G))$ denote the number of Sylow $2$-subgroups of $HO(G)$. Then $$N_2\leq N_2(G/O(G))N_2(HO(G))$$ We split into cases:
(4a) The Sylow $2$-subgroup of $G/O(G)$ is normal and $[G:HO(G)]>3$
In this case $HO(G)$ contains all Sylow $2$-subgroups. If $HO(G)$ is of index larger than $3$ then $N_2\leq \frac{|G|}{16}$ and we may argue as in case (2).
(4b) The Sylow $2$-subgroup of $G/O(G)$ is normal and $[G:HO(G)]=3$
If $HO(G)$ is of index $3$, then $HO(G)$ contains all Sylow subgroups except for the $3$-subgroups. Let $k$ be the smallest prime such that $p_k$ has an exponent of $1$ in the prime factorization of $|HO(G)|$. We can argue as in (2) to obtain that $$\sum_{p_i\neq 3}{N_{p_i}}=N(HO(G))\leq\frac{23}{50}(|G|/3)+\frac{1}{p_k-1}(|G|/3)<\frac{1}{3}|G|$$ and since $N_{3}\leq \frac{1}{3}|G|$ we have that $N\leq\frac{2}{3}|G|$.
(4c) $G/O(G)$ has a simple projective special linear group as a normal subgroup
In this case the normal subgroup of odd index given by Walter's theorem is either $PSL_2(2^m)$ with $m>1$, $PSL_2(q)$ where $q\equiv \pm 3\pmod{8}$, or either of these groups with an extra $\mathbb{Z}_2$ factor (this follows from the constraint $a_1\leq 3$). By Sylow 2-subgroups of the group $\mathrm{PSL}(2,q)$, $N_2(G/O(G))\leq\frac{1}{12}|G/O(G)|$. If $p_2>3$ or $p_2=3$ and $a_2\geq 2$, then we may argue as in case (2).
If $p_2=3$ and $a_2=1$, we can squeeze out the result by noting that $|O(G)|$ is not divisible by $3$, hence if $H'$ is a Sylow $3$-subgroup then $$N_3\leq N_3(G/O(G))N_3(H'O(G))$$ with the notation having the obvious meaning, hence since the normalizer of the $3$-subgroup has order at least $6$ we have $$\frac{N_3}{|G|}\leq\frac{N_3(G/O(G))}{|G/O(G)|}\leq \frac{1}{6}$$ so $$N\leq\sum_{p>5}{\frac{1}{p^2}}|G|+N_2+N_3+\sum_{p_i\geq 5}{N_{p_i}}\leq\frac{23}{50}|G|-\frac{1}{4}|G|-\frac{1}{9}|G|-\frac{1}{25}|G|+\frac{1}{12}|G|+\frac{1}{4}|G|+\frac{1}{6}|G|=\frac{503}{900}|G|<\frac{2}{3}|G|$$
(4d) $G/O(G)$ has a Janko group $J_1$ as a normal subgroup
In this case the normalizer of a Sylow $2$-subgroup has index $1045$, so $N_2\leq\frac{1}{1045}|G|$, which is more than small enough to apply the argument in case (2).
(4e) $G/O(G)$ has a Ree group as a normal subgroup
I couldn't find a reference on the number of Sylow $2$-subgroups of Ree groups, but note that we can deduce that $N_2\leq \frac{|G|}{24}$ because the Sylow $2$-subgroup is of order $8$ and by Burnside's transfer theorem its normalizer must have an additional factor of at least $3$, and this is enough to apply the argument in case (2).
(5) $p_1=2$, $a_2=3$ and $G$ does not have a normal $2$-complement
By the Frobenius normal $p$-complement theorem we must have that $|N_G(H)|\geq 16$ where $H$ is a Sylow $2$-subgroup, hence $N_2\leq\frac{1}{16}|G|$ and we may argue as in case (2).
(6) $p_1=2$, $a_1=1$ and $N_2=\frac{|G|}{2}$
In this case we are in the situation of Suppose that half of the elements of G have order 2 and the other half form a subgroup H of order n. Prove that H is an abelian subgroup of G., meaning that we have an abelian subgroup of index $2$, hence every Sylow $p_i$ subgroup for $i>1$ is normal, hence $$N=\frac{1}{2}|G|+n-1$$ and since in particular $n-1\leq\frac{|G|}{6}$ because $|G|\geq 2\cdot 3^{n-1}$ we have that $$N\leq \frac{2}{3}|G|$$ This bound is tight, with $S_3$ giving an example of equality (in fact $S_3$ is unique in this sense, which can be seen from the proof).
On
Here's a strongly simplified proof when the bound $2/3$ is replaced with 1 (using the same starting point as in Matt's answer).
Write $|G|=\prod_p p^{a_p}$. Write $\mathcal{P}$ for the set of primes, $\mathcal{P}_2=\{p\in \mathcal{P}:a_p\ge 2\}$ and $\mathcal{P}_2=\{p\in \mathcal{P}:a_p=1\}$ and $q=\min(p_1)$. Then the number of (nontrivial) Sylow subgroups satisfies: $$N/|G|\le \sum_{p\in\mathcal{P}_2}\frac{1}{p^2}+\frac{1}{q}$$ (where $q=\infty$, $1/q=0$ if $\mathcal{P}_1=\emptyset$).
In particular, it satisfies $$N/|G|\le \sum_{p\in\mathcal{P}}\frac{1}{p^2}+\frac{1}{(q-1)}-\frac{1}{q^2}$$ (see this post about $\sum p^{-2}$)
using that $\sum_{p\in\mathcal{P}}\frac{1}{p^2}\le 453/1000$, we see that if $q\ge 5$ then $N/|G|<453/100+1/4-1/25=663/1000<2/3$.
For $q=3$ it yields $N/|G|<46/100+1/2-1/9<85/100<1$.
For $q=2$ it only yields $N/|G|<46/100+1-1/4=121/100$, so to prove $N<|G|$ we need a little more. If $q=2$, then the signature gives a homomorphism onto the group on 2 elements, and all $p$-Sylow for odd $p$ are contained in the index 2 kernel. If $s_p$ is the set of elements of order $p$ and $N_p$ is the number of $p$-Sylow, we always have
$$\frac{N}{|G|}\le \sum_{p\in\mathcal{P}_2}\frac{N_p}{|G|}+\sum_{p\in\mathcal{P}_1}\frac{s_p}{|G|(p-1)}$$
if $q=2$ we thus have $\frac{N_p}{|G|}\le\frac{1}{2p^2}$ for all $p>2$, also the unique subgroup of index 2 consists exactly of those elements with odd order. So $s_2\le |G|/2$ and $\sum_{p>2}s_p\le |G|/2$, so $\sum_{p>2}s_p/(p-1)\le |G|/4$. Hence
$$\frac{N}{|G|}\le \frac12\left(\sum_{p\in\mathcal{P}}\frac{1}{p^2}-\frac{1}{4}\right)+\frac{3}{4}<86/100.$$ (Discussing on whether $a_3=1$ improves this to $<80/100$.)
To conclude, using nothing else than Sylow theorem, signature in the symmetric group, and an upper bound for $\sum p^{-2}$, we get $N<(86/100)|G|$.
On
I record this in case it helps anyone to produce a simpler proof. I interpret "Sylow subgroup" to mean "Non-identity Sylow subgroup". If possible, choose a finite group of minimal order subject to having strictly more than $\frac{2|G|}{3}$ Sylow subgroups. I claim that there is no prime divisor $p$ of $|G|$ such that $G$ has a normal $p$-complement. Suppose otherwise, and let $G = PN$ where $N \lhd G$ has order prime to $p$ and $ 1 \neq P \in {\rm Syl}_{p}(G)$.
Then (using minimality) $G$ has at most $\frac{2|N|}{3}$ Sylow subgroups other than Sylow $p$-subgroups, and $G$ has at most $\frac{|G|}{|P|}$ Sylow $p$-subgroups. Hence $G$ has at most $\frac{|G|}{|P|}\left( \frac{2}{3} +1\right)$ Sylow subgroups.
Hence $\frac{5}{3} \frac{|G|}{|P|} > \frac{2|G|}{3},$ so $|P| < \frac{5}{2}$ and $|P| = 2.$ Let us examine the case $|P| = 2$ more carefully
If $N_{G}(P) = P$, then $G$ has an Abelian subgroup $N$ of odd order and index $2$. It is then clear that $G$ has at most $|N| + \log_{3}(|N|)$ Sylow subgroups. This is at most $\frac{4|N|}{3}$ by elementary calculus since $|N| \geq 3.$ This contradicts the fact that $G$ has more than $\frac{2|G|}{3}$ Sylow subgroups. Hence $N_{G}(P) \neq P$, so $|N_{G}(P)| \geq 3|P|.$
Hence $G$ has at most $\frac{2|G|}{6} + \frac{|G|}{6}$ Sylow subgroups ( using minimality again for the odd Sylow subgroups), so at most $\frac{|G|}{2}$ Sylow subgroups, a contradiction.
Possibly a partial answer - we first consider a particular prime and then we prove something slightly stronger.
Let $p$ be a prime dividing the order of the group $G$. Now it is standard that $n_p:=|Syl_p(G)|=[G:N_G(P)]$. But $P \subseteq N_G(P)$ and certainly $|P| \geq p$, implying $|N_G(P)| \geq p$. Hence $n_p \leq \frac{1}{p}|G| \leq \frac{1}{2}|G|$.
Corollary Let $|G|=p^aq^b$, with $p$ and $q$ prime numbers and $p \lt q$ and $a$ and $b$ positive integers.
(1) If $p=2$ and $q \geq 7$, then the total number of Sylow subgroups of $G$ is smaller than $\frac{2}{3}|G|$.
(2) If $p$ is odd, then then the total number of Sylow subgroups of $G$ is smaller than $\frac{2}{3}|G|$.
Proof Apparently, the total number of Sylow subgroups of $G$ equals $n_p + n_q \leq (\frac{1}{p}+\frac{1}{q})|G|$. In case (1) we have $(\frac{1}{2}+\frac{1}{q})|G| \lt \frac{2}{3}|G|$, since $q \gt 6$. In case (2) $(\frac{1}{p}+\frac{1}{q})|G| \leq (\frac{1}{3}+\frac{1}{5})|G| \lt \frac{2}{3}|G|. \square$
Note: this principle can be further generalized to more prime factors, for example if $|G|=p^aq^br^c$, with $5 \leq p \lt q \lt r$, the bound of $\frac{2}{3}|G|$ still holds.