I want to show that there does not exist an automorphism of $\Bbb Z^2$ of order $5$, i.e., an element $f\in \text{Aut}(\Bbb Z^2)$ such that $f^5=\text{id}$.
Since the automorphism group of $\Bbb Z^2$ is isomorphic to the matrix group $SL_2(\Bbb Z)$, this is equivalent to showing that the group $SL_2(\Bbb Z)$ has no subgroup of order $5$, and this is indeed true (I have searched google and found that every finite subgroup of $SL_2(\Bbb Z)$ is of order a divisor of $24$).
However, I think this is making the problem harder, so I am looking for a more simple or direct approach. Any hints?
This is a good approach, when combined with linear algebra.
Suppose $M$ is a matrix in $SL_2(\Bbb Z)$ such that $M^5=I$. This means that the minimal polynomial of $M$ divides $t^5-1$. On the other hand, the characteristic polynomial of $M$ is a degree-$2$ polynomial with integer coefficients, which the minimal polynomial of $M$ also divides. In particular, every irreducible (over $\Bbb Z$) factor of the minimal polynomial of $M$ must be an irreducible (over $\Bbb Z$) factor of $t^5-1=(t-1)(t^4+t^3+t^2+t+1)$ of degree at most $2$. The only possibility is that the minimal polynomial is $t-1$ itself, which forces $M=I$.