If $G$ is a finite group, a bijection $f\colon G\to G$ is called a (normed) $\boldsymbol{\mathfrak{R}}$-automorphism if
$f$ maps subgroups of $G$ to subgroups of $G$, and
$f(gH) = f(g) f(H)$ for any $g\in G$ and $H \leq G$, i.e. $f$ maps cosets of $H$ to cosets of $f(H)$.
Any automorphism of a group is an $\mathfrak{R}$-automorphism, but the converse does not hold. For example, if $G$ is a cyclic group of prime order, then any permutation of the elements that fixes the identity is an $\mathfrak{R}$-automorphism.
In my answer to this question, I observed that the group $$ \mathbb{Z}_9 \rtimes \mathbb{Z}_3 \;=\; \langle a,b\mid a^9=b^3=1,b^{-1}ab = a^4\rangle $$ has an $\mathfrak{R}$-automorphism defined by $$ f(a^jb^k) = a^{-j}b^{-k} $$ for all $j,k$. My question is
Is there any simple way to prove that $f$ is an $\mathfrak{R}$-automorphism?
The only proof I have is by brute force: we write down all the cosets of all of the subgroups and observe that $f$ maps cosets to cosets.
More generally,
Are there any good techniques for proving that a given function is an $\mathfrak{R}$-automorphism?
For the specific example you can do some computations in the group to establish that all subgroups have to satisfy a certain property for the coset condition to be satisfied. This removes the brute force on the left cosets (but doesn't impact the rest).
Let $g=a^j b^k$ and $h=a^l b^m\in H$. Then you have $$f(gh) = f(a^{j+4^kl}b^{k+m}) = a^{-j-4^kl}b^{-k-m}.$$ We want to find $a^x b^y$ such that $f(gh)=a^{-j}b^{-k}a^{-x}b^{-y}$. Computing in $G=\mathbb Z_3\rtimes\mathbb Z_9$ we find that $y\equiv m\bmod 3$ and $x\equiv 4^{2k}l\bmod 9$.
So supposing you have shown that $f$ acts as a permutation on the set of all subgroups, then $f$ will be an $\mathfrak R$-automorphism if and only if for all subgroups $H$ and all $k,l,m\in\mathbb N$ we have $a^l b^m\in H \iff a^{4^{2k}l}b^m\in H$.
For the general case, I do not at this juncture see how to approach the problem generically, and could only use ad hoc computations for specific (types of) groups and choices of $f$.