endomorphism of finite groups

Question

Have $\mathcal{G}$ denote the set of finite groups with at least $2$ elements. How would I go about showing that if $G \in \mathcal{G}$, then $\left|\text{End}(G)\right| \le \sqrt[p]{\left|G\right|^{\left|G\right|}}$, where $\left|\text{End}(G)\right|$ is the number of endomorphisms of $G$ and $p = p(G)$ is the greatest prime divisor of $\left|G\right|$. What are the groups in $\mathcal{G}$ such that we obtain equality in the inequality? Any help appreciated!

Answer 1

Let $h$ be an element of $G$ of order $p$, let $H = \langle h \rangle = \{e, h, \dots, h^{p-1}\}$ and let $I = \{x_1, x_2, \dots, x_k\}$ be a complete set of representatives for the left cosets of $G$ mod $H$, where $x_1 = h$. It is evident that $k = |G|/p$. Every endomorphism of $G$ is uniquely determined by its values on $I$: if $x \in x_aH$, $x = x_ah^b$ for some $b$, whence $f(x) = f(x_a)f(h)^b = f(x_a)f(x_1)^b$. Hence $$|\text{End}(G)| \le |G^I| = |G|^{|G|/p} = \sqrt[p]{|G|^{|G|}}.$$Equality holds in the inequality if and only if $|\text{End}(G)| = |G^I|$, that is, if and only if any map $f: I \to G$ extends to an endomorphism of $G$. Assume that $G$ is such that previous equivalent statements hold. If $k = 1$, $|G| = p$, so $G \cong (\mathbb{Z}_p, +)$ and $|\text{End}(\mathbb{Z}_p, +)| = p = \sqrt[p]{p^p}$. Suppose now $k \ge 2$. Consider the map $f: I \to G$ given by $f(x_1) = x$, $f(x_i) = e$ for $i > 1$, any $x \in G$. We have assumed the map $f$ extends to an endomorphism of $G$, hence$$x^p = f(x_1^p) = f(e) =e.$$If $x_2^{-1} \not\in x_2H$, we can find a map $f: I \to G$, sending $x_2$ to $e$, and $x_2^{-1}$ to some other element of $G$. However, such an $f$ does not extend to an endomorphism of $G$, which goes against the assumption. Thus $x_2^{-1} \in x_2H$, or equivalently $x_2^2 \in h$. If $p$ is odd, we have that $1 - p$ is even, so $x_2 = x_2^{1-p} \in H$, again a contradiction. So we must have $p=2$ and $G$ abelian $($$x^2 = e$ for all $x \in G)$. If $k = 2$, $G = V$, the Klein four-group, which is clearly seen to satisfy the equality we want. If $k \ge 3$, $x_2x_3$ does not belong to any of the cosets $x_iH$, $i = 2, 3$, so we are able to find a map $f: I \to G$, with $f(x_2) = f(x_3) = e$ and $f(x_2x_3) \neq e$. Again we have a contradiction because we cannot extend $f$ to an endomorphism of $G$.

In conclusion, the only groups for which equality holds in our original inequality are $\mathbb{Z}_p$, where $p$ is prime, and the Klein four-group.