$\int \frac{\sqrt{3cos2x-1}}{cosx}$
ATTEMPT:
I did the following substitution
Let $sinx=t$
$cosxdx=dt$
$I=\frac{\sqrt{3(1-2t^2)-1}}{1-t^2}=\frac{\sqrt{2-6t^2}}{1-t^2}$
Now let $t=\frac{1}{z}$
$dt=-\frac{1}{z^2}dz$
Now $I=\color{red}{-} \frac{\sqrt{2z^2-6}}{z(z^2-1)}$
Lastly substitute $2z^2-6=p^2$
Finally $I=\color{red}{-}2\int \frac{p^2}{(p^2+6)(p^2+4)}dp$ which can be easily be done by using Partial fractions.
Substituting back from $p$ to $z$ to $t$ and finally to $sinx.$
I got $I=2arctan\sqrt{\frac{2-6sin^2x}{4sin^2x}}\color{red}{-}\sqrt{6}arctan\sqrt{\frac{2-6sin^2x}{6sin^2x}}$
But the answer given in the text is $\sqrt{6}arcsin(\sqrt{3}sinx)-2arctan(\frac{2sinx}{\sqrt{3Cos2x-1}})$
Where did i go wrong?
I think you are using a wrong substitution.
Try $Cos2x=1-2Sin^2x$. Then do the substitution using $t=Cosx$.