I don't yet know much about categorical limits and colimits, I have just started learning about them, and so I wanted to experiment a bit with this concept. And to that end, my first natural attempt was the following.
Let $S_n$ denote the symmetric group on n elements. Denote by $\iota_n: S_n\to S_{n+1}$ the map which sends a permutation of $[n]$ onto the associated permutation of $[n+1]$ leaving the element $n+1$ fixed. We can stick all of these maps together and then ask about the colimit of the diagram we obtain this way. My natural instinct is that it should be the symmetric group on the integers, and the maps are those which send a permutation $\tau\in S_n$ to the permutation of $\mathbb{N}$ fixing all elements bigger than $n$ and acting on $[n]\subset\mathbb{N}$ in the natural way.
Showing that this is a valid candidate for the colimit is easy, but I don't know how to show that it is in fact the colimit. I have also begun wondering if maybe this isn't the colimit, but I don't see any other obvious candidate. Any help would be appreciated, thanks.
The colimit of a countable diagram of countable groups is countable, but the full symmetric group of $\mathbb{N}$ is uncountable, so it can't possibly be the colimit. As Arturo says in the comments, the real colimit is the smaller countable group $S_{\infty}$ of permutations of finite support, meaning permutations that fix all but finitely many natural numbers.
Showing that this is the colimit is straightforward. You can show more generally that if $G$ is a group and $G_1 \subset G_2 \subset G_3 \subset \dots \subset G$ is an increasing sequence of subgroups of $G$ whose union is $G$, then $G$ is the colimit of this sequence of inclusions (this is called an increasing union). So it suffices to observe that $S_{\infty}$ has an increasing sequence of subgroups $S_n$ given by the permutations fixing all natural numbers except possibly the first $n$, and is the union of this sequence.