I have a very specific system of two trigonometric equations
$$\left( 3A^2\sin x \cos x - A \sin x \right) + \left( 3B^2\sin y \cos y - B \sin y \right) = AB \sin (x+y)$$ $$\left( 3A^2\cos^2 x - A \cos x \right) + \left( 3B^2\cos^2 y - B \cos y \right) = AB \cos (x+y) - \frac{3}{2}\left(1-A^2-B^2\right)$$
I need to solve it with respect to angles $x$, $y$ with $A$ and $B$ being parameters. If one of the parameters is equal to zero or if $A=B$ it is actually possible to solve it, but I lost all hopes to get an adequate closed-form solution in a general case.
Are there any ways to solve it? Maybe for some special values of $A$, $B$, or maybe it is provably unsolvable for specific parameters?
For the first equation,
$$\left( 3A^2\sin x \cos x - A \sin x \right) + \left( 3B^2\sin y \cos y - B \sin y \right) = AB \sin (x+y)$$
set $A=0$, you get $$ 3B^2\sin y \cos y - B \sin y = 0$$ Let $\sin y=\alpha$ $$3B^2\alpha\sqrt{1-\alpha^2}-B\alpha=0$$ $$9B^4\alpha^2(1-\alpha^2)=B^2\alpha^2$$ Now we can see one of the solutions is $\alpha=0,~\color{red}{y=n\pi}$,where $n$ is an integer.
Reduce both sides, the equation becomes $$9B^2(1-\alpha^2)-1=0$$ Using the quadratic formula $$\alpha=\frac{\pm\sqrt{4(9B^2)(9B^2-1)}}{18B^2}$$ $$\alpha=\frac{\pm\sqrt{9B^2-1}}{3B}$$ $$\color{red}{y=\sin^{-1}\left(\frac{\pm\sqrt{9B^2-1}}{3B}\right)}$$ For the second equation,
$$\left( 3A^2\cos^2 x - A \cos x \right) + \left( 3B^2\cos^2 y - B \cos y \right) = AB \cos (x+y) + \frac{3}{2}\left(1-A^2-B^2\right)$$ also set $A=0$, and you get $$\left( 3B^2\cos^2 y - B \cos y \right) = \frac{3}{2}\left(1-B^2\right)$$
Set $\cos y= \beta$ $$\left( 3B^2\beta^2 - B\beta \right) = \frac{3}{2}\left(1-B^2\right)$$ Again, use the quadratic formula $$\beta=\frac{B\pm\sqrt{B^2+18B^2(1-B^2)}}{6B^2}$$ $$\beta=\frac{1\pm\sqrt{1+18(1-B^2)}}{6B}$$ $$\color{red}{y=\cos^{-1}\left(\frac{1\pm\sqrt{19-18B^2}}{6B}\right)}$$ Since the equations you give is symmetric, I would conclude that $$\small\color{green}{x=\cos^{-1}\left(\frac{1\pm\sqrt{19-18A^2}}{6A}\right)}\large{,~}\small\color{green}{\sin^{-1}\left(\frac{\pm\sqrt{9A^2-1}}{3A}\right)}\large{,~}\small \color{green}{n\pi}$$