Let $G=S_3$ and let $H$ be the Sylow $3$-subgroup in $G$. If $\mathbb{Z}$ is the trivial module, then it can be shown that
$$H^n(H,\mathbb{Z})=\begin{cases}\mathbb{Z}&n=0\\0&n\text{ odd}\\\mathbb{Z}_3&n\text{ even}\end{cases}$$
Since $H$ is normal in $G$, $G$ acts on $H^n(H,\mathbb{Z})$ as follows. Let $g\in G$ and define $c_g:H\to H$ by $c_g(h)=ghg^{-1}$. Since $H^*(-,M)$ is contravariant, we obtain an isomorphism $c_g^*:H^n(H,\mathbb{Z})\to H^n(H,\mathbb{Z})$. Then, for $z\in H^n(H,\mathbb{Z})$, define $g\cdot z=(c_g^*)^{-1}(z)$.
There is another way to define this action on cochains. If $F\to\mathbb{Z}$ is a projective resolution over $\mathbb{Z}G$, and $f\in\operatorname{Hom}_H(F,\mathbb{Z})$, then the $G$ action on cohomology is induced by the action $(g\cdot f)(x)=gf(g^{-1}x)=f(g^{-1}x)$ (notice that the action on $\mathbb{Z}$ is trivial).
I'm trying to compute explicitly the action of $G$ on $H^n(H,\mathbb{Z})$. This question addresses my ultimate goal, which is to compute the integral cohomology of $G$, but the answer given skips over what I've asked here (it answers my question referencing some mysterious exercise AE.9, which I cannot find in Brown).
Can someone show me how $G$ acts on $H^n(H,\mathbb{Z})$, using either (or both) of the definitions given above?
A non-zero element of $H^2(H,\mathbb Z)$ is the class $\alpha$ of the non-split extension of $H$ by $\mathbb Z$ $$0\to\mathbb Z\to\mathbb Z\xrightarrow f\mathbb Z/3\to0$$ with $f(1)=1+3\mathbb Z$. Using esssentially the first description of the action (that is, a pullback construction), you can check that the transpositions of $S_3$ act on this extension by turning it into the extension with map $1\mapsto 2+3\mathbb Z$.
This means that transpositions act as $-1$ on $H^2$.
As noted above, you get the action on the whole of cohomology because $H^\bullet$ is (almost) a polynomial algebra generated by the class $\alpha$ of the above extension. Indeed, we have $H^\bullet(H,\mathbb Z)=\mathbb Z[\alpha]/(3\alpha)$ as a ring and the action of $S_3$ respects the ring structure.