@Did, what do you recommend me for?
$$\int_0^{\infty} \log\left(1-2\frac{\cos(2a)}{x^2}+\frac{1}{x^4}\right)^2 \ dx, \quad 0\le a \le \pi $$
The integration by parts is of no help. What else is left?
@Did, what do you think about the generalization? Does it have a closed form? $$\int_0^{\infty} \log\left(1-2\frac{\cos(2a)}{x^2}+\frac{1}{x^4}\right)^n \ dx, \quad 0\le a \le \pi, \space n\ge 1, n \in \mathbb{N} $$
It is unlikely that this has a closed form in general. However, I can tell you that for the special case $\cos(2a) = 0$,
$$ \int_0^\infty \log\left(1 + \frac{1}{x^4}\right)^2\ dx = \sqrt{2} \pi (6 \ln(2)+\pi)$$