For a permutation $\sigma : \mathbb{N} \to \mathbb{N}$, let $c_{\sigma}(n) = |\{k \in [n] \mid \sigma(k) \ne k\}|$, and for a function $f : \mathbb{N} \to \mathbb{R}$, let $G_f = \{\sigma \in S_{\mathbb{N}} \mid \lim_{n \to \infty} c_\sigma(n)/f(n) = 0 \}.$ For example, if $f(n) = n$, then $G_f$ is just the permutations of $\mathbb{N}$ that fix almost everything, and if $f(n) = n^2$ (or something else that grows much faster than $n$) then $G_f$ is just all permutations of $\mathbb{N}$. My question is are the groups $\{ G_{x^r} \mid r \in (0,1]\}$ all isomorphic? Isomorphic to all permutations of $\mathbb{N}$? I'm able to see that they mutually contain each other, but that isn't enough to resolve this. Have people studied groups that are defined in such ways?
This question is inspired by Is there a natural family of nonisomorphic groups parametrized by $\mathbb{R}$?, as showing my groups aren't isomorphic would resolve that question nicely. If my groups are isomorphic, I'm also curious if there are computably isomorphic in the sense that given a list of what my input permutation does I can access sequentially, I can begin to compute what the permutation it maps to does.
edit: Ycor's answer below shows that if they are isomorphic, they are conjugate, but I'm still very curious about the outcome: as mentioned in that answer, being conjugate would require a permutation with pretty surprising properties, and ruling out the existence of such a thing (or showing there is one!) seems a worthy problem in its own right. Note (as mentioned in a comment) Ycor's partial result rules out any of these groups being isomorphic to all permutations of $\mathbb{N}$
A proof that the groups $G_{x^r}$ are not conjugate, which proves that they are not isomorphic by YCor's answer:
Lemma: For any permutation $\alpha : \mathbb{N} \to \mathbb{N}$, and any $t \in (0, 1]$, there is a set $B \subset \mathbb{N}$ with $|B \cap [n]| = O(n^t)$ and $|\alpha(B) \cap [n]| = \Omega(n^t)$.
Proof: For $k \geq 0$, note that the set $A_k = \alpha^{-1}([2^{k+1}]) \setminus [2^k]$ has size at least $2^k$, so let $B_k$ be a subset of $A_k$ of size $2^{kt}$, and define $B = \bigcup_k B_k$. Then for $2^k < n \leq 2^{k+1}$, all $B_j$ with $j \geq k+1$ are disjoint from $[2^{k+1}]$, hence $$B \cap [n] \subset B \cap [2^{k+1}] \subset \bigcup_{j=0}^k B_j$$ which gives the bound $|B \cap [n]| \leq \sum_{j=0}^k 2^{jt} \leq C \cdot 2^{kt} \leq Cn^t$, meaning $|B \cap [n]| = O(n^t)$. In the other direction, for $n \geq 2$ we have $2^k \leq n < 2^{k+1}$ for some $k$, hence since $\alpha(B_{k-1}) \subset \alpha(B) \cap [2^k]$ this gives $$|\alpha(B) \cap [n]| \geq |\alpha(B) \cap [2^k]| \geq |\alpha(B_{k-1})| = 2^{(k-1)t} \geq 2^{(k+1)t - 2} \geq \frac{1}{4} n^t$$ so $|\alpha(B) \cap [n]| = \Omega(n^t)$.
Now, suppose there is an isomorphism $f$ from $G_{x^s}$ to $G_{x^r}$, for some $0 < r < s \leq 1$. By YCor's answer, $f$ takes the form $f(g) = \alpha g \alpha^{-1}$ for some permutation $\alpha : \mathbb{N} \to \mathbb{N}$. Take $t$ with $r < t < s$, and let $B \subset \mathbb{N}$ be as in the Lemma. Now let $\pi$ be a permutation $\mathbb{N} \to \mathbb{N}$ with $\{k \in \mathbb{N} \mid \pi(k) \neq k\} = B$; it is straightforward to construct such a permutation. We have $c_\pi(n) = |B \cap [n]| = O(n^t) = o(n^s)$, so $\pi \in G_{x^s}$, and thus $\sigma := f(\pi)$ is in $G_{x^r}$. But $\sigma = \alpha \pi \alpha^{-1}$, so $\{k \in \mathbb{N} \mid k \neq \sigma(k)\} = \{\alpha(k) \mid k \neq \pi(k)\} = \alpha(B)$, so $c_\sigma(n) = |\alpha(B) \cap [n]| = \Omega(n^t)$, which contradicts $\sigma \in G_{x^r}$. Thus $G_{x^r}$ and $G_{x^s}$ are not isomorphic.