Given an arbitrary constant 3-D rotation matrix $\mathbf{R}$ and a 3-D vector field $\mathbf{A}$, how can I find the vector field $\mathbf{B}$ such that $$\nabla\times\mathbf{B}=\mathbf{R}\left(\nabla\times\mathbf{A}\right)$$ ?
I followed two ways, without really finding a way out
I wrote the LHS and RHS explicitly. If I write $\mathbf{R}$ as the matrix $$\mathbf{R} = \begin{pmatrix}a & b & c\\ d & e & f\\ g & h & i \end{pmatrix}$$ (which is obviously an overkill, since the actual degrees of freedom of an arbitrary rotation are just three), then my equation looks like the nasty \begin{align*} \partial_{y}B_{z}-\partial_{z}B_{y} & =a\left(\partial_{y}A_{z}-\partial_{z}A_{y}\right)+b\left(\partial_{z}A_{x}-\partial_{x}A_{z}\right)+c\left(\partial_{x}A_{y}-\partial_{y}A_{x}\right)\\ \partial_{z}B_{x}-\partial_{x}B_{z} & =d\left(\partial_{y}A_{z}-\partial_{z}A_{y}\right)+e\left(\partial_{z}A_{x}-\partial_{x}A_{z}\right)+f\left(\partial_{x}A_{y}-\partial_{y}A_{x}\right)\\ \partial_{x}B_{y}-\partial_{y}B_{x} & =g\left(\partial_{y}A_{z}-\partial_{z}A_{y}\right)+h\left(\partial_{z}A_{x}-\partial_{x}A_{z}\right)+i\left(\partial_{x}A_{y}-\partial_{y}A_{x}\right) \end{align*} which intimidates me quite a bit... Rearranging it acquires an interesting structure, \begin{align*} \partial_{y}B_{z}-\partial_{z}B_{y} & =\partial_{x}\left(cA_{y}-bA_{z}\right)+\partial_{y}\left(aA_{z}-cA_{x}\right)+\partial_{z}\left(bA_{x}-aA_{y}\right)\\ \partial_{z}B_{x}-\partial_{x}B_{z} & =\partial_{x}\left(fA_{y}-eA_{z}\right)+\partial_{y}\left(dA_{z}-fA_{x}\right)+\partial_{z}\left(eA_{x}-dA_{y}\right)\\ \partial_{x}B_{y}-\partial_{y}B_{x} & =\partial_{x}\left(iA_{y}-hA_{z}\right)+\partial_{y}\left(gA_{z}-iA_{x}\right)+\partial_{z}\left(hA_{x}-gA_{y}\right) \end{align*} but I don't really know how to continue.. I don't think that replacing the actual rotation matrix elements would even help..
Then I tried a different, let's say geometric, way. I Fourier transformed both sides of my equation and, as (1) the Fourier transform of the $\nabla$ operator is $i\mathbf{k}$ and (2) the Fourier transform commutes with a rotation matrix, I obtained $$i\mathbf{k}\times\tilde{\mathbf{B}}=\mathbf{R}\left(i\mathbf{k}\times\tilde{\mathbf{A}}\right)$$ which is a nice expression involving just vectors and not nasty partial derivatives. Now, I can rewrite this as $$\mathbf{k}\times\tilde{\mathbf{B}}=\mathbf{R}\left(\mathbf{k}\right)\times\mathbf{R}\left(\tilde{\mathbf{A}}\right)$$ but this expression, although simple and nice, does not really help me find $\mathbf{B}$...
Perhaps I am just rusted and I am missing something simple....
I think the reason you're having trouble is that there may be no such vector field $B$. Let's take an example:
$$A(x, y, z) = \pmatrix{z\sin x + y \cos x\\ 0\\ 0} $$ Then $$ curl ~ A(x, y, z) = \pmatrix{0 \\ \sin x \\ -\cos x} $$ if I've done the computation right. And notice that the divergence of that is $0$, because the divergence of the curl of any vector field on a contractible set like $\Bbb R^3$ is always zero (at least if everything is $C^2$, which it certainly is in this case).
Now let $$ R = \pmatrix{0 & 1 & 0 \\ -1 & 0 & 0 \\ 0 & 0 & 1}. $$ Then let's apply $R$ to the curl of $A$ and give it a name: $$ Q = R(\nabla \times A) = \pmatrix{0 & 1 & 0 \\ -1 & 0 & 0 \\ 0 & 0 & 1}\pmatrix{0 \\ \sin x \\ -\cos x} = \pmatrix{\sin x \\ 0 \\ -\cos x}. $$ Now the divergence of $Q$ is $$ \nabla \cdot (R(\nabla \times A)) = \cos x + 0 + 0 = \cos x, $$ which is not zero. That means that $Q$ cannot be the curl of any vector field. And that's why you couldn't find one. :)