According to convolution theorem convolution between two functions $f(r), g(r)$ in real space (e.g. 2D,3D) can be calculated as product of Fourier images.
Convolution $(f*g)(\vec R) = \int_{\vec r} f(\vec r)g(\vec R-\vec r)$ is an integral over product of a function $f(\vec r)$ and a flipped translation of function $g(\vec R-\vec r)$.
I would like something similar for rotation: $(f \circ g)(\alpha) = \int_{\vec r} f(\vec r)g( \hat R(\alpha).\vec r)$
Where $R(\alpha)$ is a rotation matrix which rotates coordinate $r$ by angle $\alpha$.
E.g. in 2D space:
$$ \hat R(\alpha) = \begin{bmatrix} \cos(\alpha) -\sin(\alpha) \\ \sin(\alpha) \cos(\alpha) \end{bmatrix} $$
EDIT: There is a paper about using FFT to rotate images, but I'm not quite sure how to apply it to evaluation of rotational-convolution
For what it's worth, in the 2D case you could rewrite the integral in polar coordinates as
$$\int_\rho\int_\theta f(\rho,\theta)g(\rho,\theta+\alpha)\rho\,d\rho\,d\theta$$ and see this as an integral of convolutions over $\theta$,
$$\int_\rho f(\rho)\star g(\rho)\,\rho\,d\rho$$