Informally: Every group is representable (up to an isomorhism) as a group of isomorphisms.
Formally: For every group $G$ there exists a binary relation $f$ on some set $U$ such that $G$ is isomorphic to group of all bijections $g$ on $U$ such that $g^{-1}\circ f\circ g=f$.
In other words, is every group $G$ isomorphic to the group of isomorphisms of some (possibly infinite) digraph?
Moreover, is this digraph $f$ unique up to an isomorphism, for each group $G$?
It appears that the answer to your question is yes, every group $G$ arises as the automorphism group of some graph. In fact, we can even ask for that graph to be undirected.
The case of a finite group being representable as the automorphism group of a finite undirected graph is called Frucht's theorem.
This has apparently been generalised to infinite groups, as Wikipedia briefly discusses in this section. The references given for this result are
I don't have access to either, and I don't claim to have read or understood these proofs!
For the uniqueness part, Wikipedia says that in the case of finite groups, uniqueness fails pretty catastrophically, and indeed it's easy enough to come up with examples like Jair mentions in the comments. In general the complement of a graph has the same automorphism group, for example. Wikipedia doesn't say anything about uniqueness in the case of infinite groups but it would be surprising if there was ever this kind of uniqueness.
In fact, one can see via fairly elementary means that the automorphism group never determines a graph. Suppose $G$ is a graph with at least two vertices. Add a new vertex $v$, and an edge between $v$ and every old vertex. Also add a vertex $u$ and $n \ge 1$ vertices between $u$ and $v$ forming a linear graph.
Observe that $u$ is uniquely determined by the property "$u$ has degree $1$ and its neighbour has degree $2$", and hence the path between $u$ and $v$ is fixed by any automorphism. It follows that this new graph has the same automorphism group as $G$, and that each different choice of $n \ge 1$ gives a different isomorphism type. They're also all connected! If you add some kind of transitivity hypothesis of the sort Travis Willse mentions, you may have to think of a cleverer argument than this.