How do we show that the given result by Mathematica is correct?
$$\int_{0}^{\infty}{\cos(x^n)-\cos(x^{2n})\over x}\cdot{\ln{x}}\mathrm dx={12\gamma^2-\pi^2\over 2(4n)^2}\tag1$$ $n>0$
Where $\gamma=0.577216...$
I would try substitution, because it may help to simplify the problem into a manage integral to deal with.
$u=x^n$
$du=nx^{n-1}dx.$
$${1\over n}\int_{0}^{\infty}{\cos(u)-\cos(u^2)\over u^{1\over n}}\cdot{\ln{u^{1\over n}}}{\mathrm dx\over u^{n-1\over n}}={12\gamma^2-\pi^2\over 2(4n)^2}$$
Simplified to
$${1\over n^2}\int_{0}^{\infty}{\cos(u)-\cos(u^2)\over u}\cdot{\ln{u}}\mathrm du={12\gamma^2-\pi^2\over 2(4n)^2}$$
We can remove $\ln{u}$ by doing another substitution
$v=\ln{u}$
$udv=du$
$${1\over n^2}\int_{-\infty}^{\infty}{\cos(e^v)-\cos(e^{2v})\over e^v}\cdot{v}\cdot{e^v}\mathrm du={12\gamma^2-\pi^2\over 2(4n)^2}$$
Then we finally simplified to
$$={1\over n^2}\int_{-\infty}^{\infty}v\cos(e^v)\mathrm dv -{1\over n^2}\int_{-\infty}^{\infty}v\cos(e^{2v})\mathrm dv$$
At this stage I would apply integration by parts but it seems to show a problem for me to do. So I need some help. Thank you.
Define $$\mathcal{I}=\int_{0}^{\infty }\frac{\cos x-\cos x^2}{x}\, \ln x\, \mathrm{d}x$$ and $$\mathcal{I}\left ( \alpha \right )=\int_{0}^{\infty }x^{\alpha-1}\left ( \cos x-\cos x^{2} \right )\mathrm{d}x$$ and $\mathcal{I}\left(0\right)=-\dfrac{\gamma}{2} $, see a proof here.
Now let's see the integral below
Proof: Let $\displaystyle r = a/b \in (0, 1)$. Then \begin{align*} &\int_{0}^{\infty} x^{a-1} \cos(x^{b}) \, \mathrm{d}x = \frac{1}{b} \int_{0}^{\infty} \frac{\cos t}{t^{1-r}} \, \mathrm{d}t = \frac{1}{b \Gamma(1-r)} \int_{0}^{\infty} \left( \int_{0}^{\infty} u^{-r} e^{-tu} \, \mathrm{d}u \right) \cos t \, \mathrm{d}t \\ &= \frac{1}{b \Gamma(1-r)} \int_{0}^{\infty} \left( \int_{0}^{\infty} e^{-ut} \cos t \, \mathrm{d}t \right) u^{-r} \, \mathrm{d}u = \frac{1}{b \Gamma(1-r)} \int_{0}^{\infty} \frac{u^{1-r}}{u^{2} + 1} \, \mathrm{d}u \\ &= \frac{1}{b \Gamma(1-r)} \int_{0}^{\frac{\pi}{2}} \tan^{1-r} \theta\,\mathrm{d}\theta \end{align*} Simplifying using the beta function identity, we have \begin{align*} \int_{0}^{\infty} x^{a-1} \cos(x^{b}) \, \mathrm{d}x &= \frac{\Gamma\left(1-\dfrac{r}{2}\right)\Gamma\left(\dfrac{r}{2}\right)}{2b \Gamma(1-r)\Gamma(r)} \, \Gamma(r)= \frac{ \sin (\pi r)}{2b \sin \left(\dfrac{\pi r}{2}\right)} \, \Gamma(r) = \frac{1}{b} \Gamma(r) \cos \left( \frac{\pi r}{2} \right)\\ &=\frac{1}{b}\cos\left(\frac{\pi a}{2b}\right)\Gamma \left(\frac{a}{b} \right) \end{align*} Hence $$\mathcal{I}\left ( \alpha \right )=\Gamma \left ( a \right )\cos\left ( \frac{a\pi }{2} \right )-\frac{1}{2}\Gamma \left ( \frac{a}{2} \right )\cos\left ( \frac{a\pi }{4} \right )$$ Using
Proof: \begin{align*} \lim_{x\to 0}x\Gamma(x)=\lim_{x\to 0}\Gamma(1+x)=1 \end{align*} \begin{align*} \lim_{x\to 0}\Gamma(x)-\frac{1}{x}&=\lim_{x\to 0}\frac{x\Gamma(x)-1}{x}\\ &=\lim_{x\to 0}\Gamma(x+1)\psi (x+1)=-\gamma\\ \end{align*} \begin{align*} \lim_{x\to 0}\frac{1}{x}\left(\Gamma(x)-\frac{1}{x}+\gamma\right)&=\lim_{x\to 0}\frac{\Gamma(x+1)-1+\gamma x}{x^2}\\ &=\lim_{x\to 0}\frac{\Gamma(x+1)\psi(x+1)+\gamma }{2x}\\ &=\lim_{x\to 0}\frac{\Gamma(x+1)\psi^2(x+1)+\psi'(x+1)\Gamma(x+1)}{2}\\ &=\frac{1}{2}\left(\gamma^2+\frac{\pi^2}{6}\right) \end{align*}
So, we have \begin{align*} \mathcal{I}&=\mathcal{I}'\left(0\right)\\ &=\lim_{a\rightarrow 0}\frac{\mathcal{I}\left(a\right)-\mathcal{I}\left(0\right)}{a-0}\\ &=\frac{-\dfrac{1}{2}\cos\dfrac{a\pi}{4}\Gamma\left(\dfrac{a}{2}\right)+\cos\dfrac{a\pi}{2}\Gamma\left(a\right)+\dfrac{\gamma}{2}}{a}\\ &=\lim_{a\rightarrow 0}\frac{-\dfrac{1}{2}\left(1-\dfrac{a^2\pi^2}{32}+o\left(a^2\right)\right)\left(\dfrac{2}{a}-\gamma +\dfrac{1}{12}\left(6\gamma^2+\pi^2\right)\dfrac{a}{2}+o\left(a\right)\right)+\left(1-\dfrac{a^2\pi^2}{8}+o\left(a^2\right)\right)\left(\dfrac{1}{a}-\gamma +\dfrac{1}{12}\left(6\gamma^2+\pi^2\right)a+o\left(a\right)\right)+\dfrac{\gamma}{2}}{a}\\ &=\lim_{a\rightarrow 0}\frac{\dfrac{a\pi^2}{32}-\dfrac{a}{48}\left(6\gamma^2+\pi^2\right)-\dfrac{a\pi^2}{8}+\dfrac{a}{12}\left(6\gamma^2+\pi^2\right)+o\left(a\right)}{a}\\ &=\frac{3\gamma^2}{8}-\frac{\pi^2}{32} \end{align*} Hence $$\int_{0}^{\infty}{\cos x^n-\cos x^{2n}\over x}\, {\ln{x}}\, \mathrm dx={1\over n^2}\int_{0}^{\infty}{\cos x-\cos x^2\over x}\,{\ln x}\, \mathrm dx={12\gamma^2-\pi^2\over 2(4n)^2}$$