How do we solve this integration? $$\int\cos(2 x) \sqrt{\sin^2(x) + 1}\, dx.$$
I used WolframAlpha, the result is $$\frac{1}{6}\left(8 \,F(x\mid-1) - 6\, E(x\mid-1) + \sin(2 x) \sqrt{6 - 2 \cos(2x)}\,\right) + C.$$
But I don't know how to get that within the steps.
I hope to get some help from you, please.
This is definitely a case where the process is smoother when you already know the goal. First a few preliminaries with regards to WolframAlpha syntax:
The Elliptic Integral of the First Kind is given by $$F(x\mid m) = \int_{0}^{x}\frac{d\theta}{\sqrt{1-m\sin^{2}(\theta)}}$$ and the Elliptic Integral of the Second Kind is given by $$E(x\mid m) = \int_{0}^{x}\sqrt{1 - m\sin^{2}(\theta)}\,d\theta.$$
We will also make use of several trigonometric identities, in particular \begin{align*} \cos(2x) = 1-2\sin^{2}(x)\\ \sin(2x) = 2\sin(x)\cos(x). \end{align*}
It is also worth noting that with the double-angle identity for $\cos$ we have that $$\sqrt{6 - 2 \cos(2x)} = \sqrt{6 - 2 + 4\sin^2(x)} = \sqrt{4 + 4\sin^{2}(x)} = 2\sqrt{1+\sin^{2}(x)}.$$
So, we will show that $$\int\cos(2 x) \sqrt{1+\sin^{2}(x)}\, dx = \frac{1}{6}\left(8 \,F(x\mid-1) - 6\, E(x\mid-1) + 2\sin(2 x) \sqrt{1+\sin^{2}(x)}\,\right) + C.$$
To this end, we have \begin{align*} \cos(2 x) \sqrt{1+\sin^{2}(x)} &= (1 - 2\sin^{2}(x))\sqrt{1+\sin^{2}(x)}\\[5pt] &=\sqrt{1+\sin^{2}(x)} - 2\sin^{2}(x)\sqrt{1+\sin^{2}(x)}\\[5pt] &=\sqrt{1+\sin^{2}(x)} - \frac{2\sin^{2}(x)}{\sqrt{1+\sin^{2}(x)}} - \frac{2\sin^{4}(x)}{\sqrt{1+\sin^{2}(x)}}\\[5pt] &=\sqrt{1+\sin^{2}(x)} - 2\sqrt{1+\sin^{2}(x)} + \frac{2}{\sqrt{1+\sin^{2}(x)}} - \frac{2\sin^{4}(x)}{\sqrt{1+\sin^{2}(x)}}\\[5pt] &=-\sqrt{1+\sin^{2}(x)}+\frac{2}{\sqrt{1+\sin^{2}(x)}} - \frac{2\sin^{4}(x)}{\sqrt{1+\sin^{2}(x)}}. \end{align*} Note that here we used $$-\frac{2\sin^{2}(x)}{\sqrt{1+\sin^{2}(x)}} = -\frac{2(1+\sin^{2}(x) - 1)}{\sqrt{1+\sin^{2}(x)}} = -\frac{2(1+\sin^{2}(x))}{\sqrt{1+\sin^{2}(x)}} + \frac{2}{\sqrt{1+\sin^{2}(x)}}.$$
Continuing, then, we have \begin{align*} \cos(2 x) \sqrt{1+\sin^{2}(x)} &=-\sqrt{1+\sin^{2}(x)}+\frac{2}{\sqrt{1+\sin^{2}(x)}} - \frac{2\sin^{4}(x)}{\sqrt{1+\sin^{2}(x)}}\\[5pt] &=-\frac{d}{dx}E(x\mid -1)+ 2\frac{d}{dx}F(x\mid -1)- \frac{2\sin^{4}(x)}{\sqrt{1+\sin^{2}(x)}}. \end{align*} So, we will focus on the last term above, doing some adding/subtracting/manipulating until we get something we recognize: \begin{align*} \frac{2\sin^{4}(x)}{\sqrt{1+\sin^{2}(x)}} &= \frac{2}{3\sqrt{1+\sin^{2}(x)}}\left[3\sin^{4}(x)\right]\\[5pt] &=\frac{2}{3\sqrt{1+\sin^{2}(x)}}\biggl[1 - 1 + 2\sin^{2}(x) - \sin^{2}(x)+2\sin^{4}(x) - \sin^{2}(x) + \sin^{4}(x)\biggr]\\[5pt] &=\frac{2}{3\sqrt{1+\sin^{2}(x)}}\biggl[1 - \bigl(1 - 2\sin^{2}(x)\bigr) - \bigl(1 - 2\sin^{2}(x)\bigr)\sin^{2}(x) - \bigl(1 - \sin^{2}(x)\bigr)\sin^{2}(x)\biggr]\\[5pt] &=\frac{2}{3\sqrt{1+\sin^{2}(x)}}\biggl[1 - \cos(2x) - \cos(2x)\sin^{2}(x) - \cos^{2}(x)\sin^{2}(x)\biggr]\\[5pt] &=\frac{1}{3\sqrt{1+\sin^{2}(x)}}\biggl[2 - 2\cos(2x)\bigl(1+ \sin^{2}(x)\bigr) - \sin(2x)\sin(x)\cos(x)\biggr]\\[5pt] &=\frac{2}{3\sqrt{1+\sin^{2}(x)}} - \frac{2\cos(2x)\bigl(1+ \sin^{2}(x)\bigr)}{3\sqrt{1+\sin^{2}(x)}} - \frac{\sin(2x)\sin(x)\cos(x)}{3\sqrt{1+\sin^{2}(x)}}\\[5pt] &=\frac{2}{3\sqrt{1+\sin^{2}(x)}} - \frac{2}{3}\cos(2x)\sqrt{1+\sin^{2}(x)} -\frac{\sin(2x)\sin(x)\cos(x)}{3\sqrt{1+\sin^{2}(x)}}\\[5pt] &=\frac{2}{3}\frac{d}{dx}F(x\mid -1)-\frac{1}{3}\frac{d}{dx}\sin(2x)\sqrt{1+\sin^{2}(x)}. \end{align*}
All together, then, we have $$ \cos(2 x) \sqrt{1+\sin^{2}(x)} = -\frac{d}{dx}E(x\mid -1)+ \frac{4}{3}\frac{d}{dx}F(x\mid -1)+\frac{1}{3}\frac{d}{dx}\sin(2x)\sqrt{1+\sin^{2}(x)}, $$
which upon integrating gives us our desired result: $$\int\cos(2 x) \sqrt{1+\sin^{2}(x)} \,dx= -E(x\mid -1) + \frac{4}{3}F(x\mid -1) + \frac{1}{3}\sin(2x)\sqrt{1+\sin^{2}(x)} + C$$
or upon factoring a $1/6$: $$\int\cos(2 x) \sqrt{1+\sin^{2}(x)}\, dx = \frac{1}{6}\left(8 \,F(x\mid-1) - 6\, E(x\mid-1) + 2\sin(2 x) \sqrt{1+\sin^{2}(x)}\,\right) + C.$$