I think it was about 15 years ago that Igor Pak told me that the symmetric group on six elements has outer automorphisms, and I was startled. Somehow that had escaped my notice. That one is a freak: no other finite symmetric group has any outer automorphisms.
The only other assertion of the existence of a finite mathematical object that has caused me to feel similarly instantly startled was in a talk by Nathan Williams that ended with a description of rotational symmetries in Young's lattice. Unlike the example in the paragraph above, this one wouldn't be particularly interesting if it were an isolated case rather than an infinite sequence of cases.
What are the best examples of "anomalous" finite objects in mathematics?
I recall a comment somewhere (can't remember the source now) along the lines that the existence of an outer automorphism of $S_6$ is "responsible" for many other more complicated anomalies in finite group theory. This includes the Mathieu groups $M_{12}$, $M_{24}$, of which the latter is used to build the Conway group $Co_1$, which is used to build the Monster group.
To give one example of this, recall the fact that $M_{24}$ is the symmetry group of a $(5,8,24)$ Steiner system: a collection of subsets of $X=\{1,2,\ldots,24\}$ of size 8 (called octads) such that every subset of $X$ of size 5 lies in a unique octad. In this form, $M_{24}$ is a subgroup of $S_{24}$.
Now consider 2 octads $O, O'$ such that $|O \cap O'| = 2$. (These exist because of the "Leech triangle".) Let $G$ be the subgroup $M_{24}$ which fixes (setwise) both $O$ and $O'$. We can show that the action of $G$ on $X$ decomposes into orbits of size 6 + 6 + 10 + 2.
Let $X_1$ and $X_2$ be the two orbits of size 6. By restricting the action of $G$ to $X_i$, we get homomorphisms $\phi_i : G \to \operatorname{Sym}(X_i)$. We can show that the $\phi_i$ are isomorphisms. Thus we get a map $\operatorname{Sym}(X_1) \to \operatorname{Sym}(X_2)$. By choosing an arbitrary bijection of each $X_i$ with $\{1,2,3,4,5,6\}$, we then get a map $S_6 \to S_6$, and we can check that this gives an outer automorphism of $S_6$. (Such explicit calculations with $M_{24}$ are (relatively) easy to do with Conway's Miracle Octad Generator.)
This construction also gives a transitive subgroup of $S_{10}$ isomorphic to $S_6$, which is probably also anomalous.