I would like to evaluate this integral
$$\large \int_{0}^{\pi/2}\cos(2x)\ln^2(\tan(x/2))\mathrm dx$$
Choosing
$u=\frac{x}{2}$
$\mathrm dx=2\mathrm du$
$$2\int_{0}^{\pi/4}\cos(4u)\ln^2(\tan u)\mathrm du$$
Choosing $\cos(4u)=\sin^4(u)-6\cos^2(u)\sin^2(u)+\cos^4(4)$
transforming into
$$2\int_{0}^{\pi/4}\sec^2(u)\cdot \frac{[\tan^2(u)-2\tan(u)-1][\tan^2(u)+2\tan(u)-1]\ln^2(\tan(u))}{(1+\tan^2(u))^3}du$$
Choosing
$v=\tan(u)$
$\mathrm du=\frac{1}{\sec^2(u)}\mathrm dv$
$$2\int_{0}^{\infty}\frac{(v^2-2v-1)(v^2+2v-1)\ln^2(v)}{(1+v^2)^3}\mathrm dv$$
$$2\int_{0}^{\infty}\frac{\ln^2(v)}{1+v^2}dv-4\int_{0}^{\infty}\frac{\ln^2(v)}{(1+v^2)^2}dv+4\int_{0}^{\infty}\frac{\ln^2(v)}{(1+v^2)^3}dv$$
I was thinking of applying binomial series, but it can't be don't
how would I continue this?
Considering $$I=\int\cos (2 x) \log ^2\left(\tan \left(\frac{x}{2}\right)\right)\,dx$$ use integration by parts $$u=\log ^2\left(\tan \left(\frac{x}{2}\right)\right)\implies du=2 \csc (x) \log \left(\tan \left(\frac{x}{2}\right)\right)\,dx$$ $$dv=\cos(2x)\,dx \implies v=\frac{1}{2} \sin (2 x)$$ $$I=\frac{1}{2} \sin (2 x) \log ^2\left(\tan \left(\frac{x}{2}\right)\right)-2\int \cos (x) \log \left(\tan \left(\frac{x}{2}\right)\right)\,dx$$ One more integration by parts to get the antiderivative and a beautiful result for the integral.