Where extension is: https://en.wikipedia.org/wiki/Group_extension.
We have that, $\textrm{Aut}(\mathbb{Z}_{4}) = \mathbb{Z}_{2}$. We have that, $\textrm{Aut}(\mathbb{Z}_4) = \{f,g\}$, where: $f(1) = 1$ and $g(1) = 3$.
We must now find all homomorphisms from $\mathbb{Z}_{3}$ to $\textrm{Aut}(\mathbb{Z}_{4})$. The number of such homomorphisms would be the number of Homs from $\mathbb{Z}_3$ to $\mathbb{Z}_2$, which is equal to $\gcd(3,2) = 1$. So we only have the trivial hom which is $\alpha(x) = f$ for all $x \in \mathbb{Z}_3$. We now reference this part of Dummit and Foote:
We have that, $A^{G} = \mathbb{Z}_4$, because every point is a fixed point with regards to the only hom $\alpha$ (the action induced by it). Now here is my confusion, what do they mean by $NA$ in this text since I'm trying to compute, $$H^{2}(\mathbb{Z}_3,\mathbb{Z}_4)?$$
Thanks!
The $N$ stands for Norm. If $G$ is finite. We define for a $G$-Module $A$ the norm $$N: A \to A, x \mapsto \sum_g gx.$$ Here we resolve $A=\mathbb{Z}$ with the above complex. Since (by convention) every group acts trivially on $\mathbb{Z}$ we have $A^G=A$. And the norm is just multiplication by the cardinality of $G$. So $$H^2(G,A)=A^G/NA=\mathbb{Z}/\# G \mathbb{Z}. $$
Remark: it is not a coincidence that the cohomology groups are cyclic. All these formulae follow from the machinery of Tate-Cohomology (see e.g. Neukirch-Schmidt-Wingberg, Cohomology of Number Fields).