If a group $G$ has $|G| = n$ elements, when does $\text{Aut}(G)$ have $n!$ elements?

Question

For some problems like this one, we know that $\text{Aut}(G)$ has exactly $n!$ elements, but in other problems like this one, $\text{Aut}(G)$ has less than $n!$ elements. How do you know in general how many elements $\text{Aut}(G)$ has? Like why in the first link we know exactly $n!$ elements without thinking, but in the second link we had to analyze to determine that $|\text{Aut}(G)| \neq |\text{Sym}(G)|$?

Answer 1

So there's an answer (although I'm by no means an expert):

In the first linked question, $X$ was just a set, and so its automorphism group was every single permutation of $X$, no questions asked.

This is generally not the case, although $n!$ is an upper bound for the number of automorphisms of a finite group of order $n$ (Since automorphisms are bijections, just with added structure).