Mathematica is able to solve the indefinite integral
$$\int\ln(a\sin(2x)+b)\cos(x)dx$$
analytically for $b>a>0$, but the resulting $\arctan$ terms are too complicated for the next steps I need to perform. I assume these $\arctan$ terms are the result of Weierstrass substitution, so I am wondering if this integral can be evaluated in a different way, maybe by substitution, so there are no complicated $\arctan$ terms in the result?
Thanks!
It looks much better if you write $$\int\log(a\sin(2x)+b)\cos(x)\,dx=$$ $$\int\log(\sin(2x)+(c^2-1))\cos(x)\,dx+\log(a)\int \cos(x)\,dx$$ with $c^2-1=\frac b a$ specifying $c>1$ and using the FullSimplify command.
Simplifying the arctangents and the logarithms, you should end for the first integral with $$-\sqrt{c^2-2} \tan ^{-1}\left(\frac{\sqrt{c^2-2} (\sin (x)+\cos (x)+1)}{2-c^2+\sin(x)+\cos (x)}\right)+$$ $$\frac c 2 \log \left(-\frac{c+\sin (x)-\cos (x)}{c-\sin (x)+\cos (x)}\right)+\sin (x) \left(\log \left(c^2+\sin (2 x)-1\right)-2\right)$$