Theorem 5.25 in Apostol's Analytic Number Theory book (Wolstenholme's theorem) states that the following holds for any prime $p \ge 5$ : $$\sum_{k=1}^{p-1} \dfrac{(p-1)!}{k} = 0 \mod p^2.$$ I read and read the proof but I never found somewhere when $p=2, 3$ are invalid. That is, in the process of proof there is nowhere when the proof fails for $p=2, 3$. However, the statement does not hold for $p=2, 3$ and holds for $p=5$. Perhaps in order for $S_1$ to appear in $g(x)$, $p=2$ has been put aside, but $S_{p-2}=S_1$ for $p=3$ still is valid. So how is the proof valid for $p=2,3$ as well but the Wolstenholme's identity is not valid for $p=2,3$?
The issue is just before the end of the proof, where an equation is taken modulo $p^3$. If $p \ge 5$, then this will cancel all terms except $-S_{n-1}p$, but if $p=3$, then $p^{p-1} = 9$ is not divisible by $p^3 = 27$ and will not be cancelled either.
In that case, you get $3S_1 \equiv 3^2 = 9$ mod 27, so $S_1 \equiv 3$ mod 9, which is confirmed by the actual sum being $S_1 = 3$.