How do I find the value of this integral?
$$I=\int_{0}^{\Large\frac{\pi}{4}} \ln {(\sin x)}\cdot\ln {(\cos x)} \left(\dfrac{\ln{(\sin x)}}{\cot x}+\dfrac{\ln {(\cos x)}}{\tan x}\right)dx$$
I tried substituting $t=\ln {(\sin x)}\cdot \ln {(\cos x)}$ and $t=\dfrac{\ln {(\sin x)}}{\ln {(\cos x)}}$, but it isn't working.
On
Here I use the Beta function to evaluate this integral and the calculation is a little bit tedious. As Omran Kouba did, \begin{eqnarray} I&=&\int_{0}^{\Large\frac{\pi}{2}} \ln^2 {(\sin x)}\cdot\ln {(\cos x)}\tan xdx. \end{eqnarray} Let $$ J(a,b)=\int_0^{\frac{\pi}{2}}\sin^ax\cos^bx\tan xdx. $$ Then clearly $$ \frac{\partial^3 J(0,0)}{\partial a^2\partial b}=I.$$ Now, by the change of variables $\sin x\to t$ and then $t^2\to u$, it is easy to see \begin{eqnarray} J(a,b)=\frac{\Gamma(1+\frac{a}{2})\Gamma(\frac{b}{2})}{2\Gamma(1+\frac{a+b}{2})}. \end{eqnarray} Noting $\Gamma'(x)=\Gamma(x)\psi_0(x), \Gamma''(x)=\Gamma(x)\psi_0^2(x)+\Gamma(x)\psi_1(a)$, we have \begin{eqnarray} \frac{\partial^2 J(a,b)}{\partial a^2}&=&\frac{\Gamma(1+\frac{a}{2})\Gamma(\frac{b}{2})}{8\Gamma(1+\frac{a+b}{2})}[\psi_0^2(1+\frac{a}{2})-2\psi_0(1+\frac{a}{2})\psi_0(1+\frac{a+b}{2})\\ &&+\psi_0^2(1+\frac{a+b}{2})+\psi_1(1+\frac{a}{2})-\psi_1(1+\frac{a+b}{2})]. \end{eqnarray} Using $$ \Gamma(1)=1, \psi_0(1)=\gamma, \psi_1(1)=\frac{\pi^2}{6}, $$ we obtain $$ \frac{\partial^2 J(0,b)}{\partial a^2}=\frac{\Gamma(\frac{b}{2})}{48\Gamma(1+\frac{b}{2})}(6\gamma^2+\pi^2+12\gamma\psi_0(1+\frac{b}{2})+6\psi_0^2(1+\frac{b}{2})-6\psi_1(1+\frac{b}{2}))) $$ and hence \begin{eqnarray} \frac{\partial^3 J(0,b)}{\partial a^2\partial b}&=&\frac{\Gamma(\frac{b}{2})}{96\Gamma(1+\frac{b}{2})}[-6\psi_0^3(1+\frac{b}{2})+6\psi_0^2(1+\frac{b}{2})(-2\gamma+\psi_0(1+\frac{b}{2}))\\ &&+\psi_0(\frac{b}{2})(6\gamma^2+\pi^2-6\psi_1(1+\frac{b}{2}))+12\gamma\psi_1(1+\frac{b}{2})+\gamma\psi_0(1+\frac{b}{2})(-6\gamma^2-\pi^2\\ &&+12\gamma\psi_0(\frac{b}{2})+18\gamma\psi_1(1+\frac{b}{2}))-6\gamma\psi_2(1+\frac{b}{2})). \end{eqnarray} Noting, for $z\in(0,1)$, $$ \Gamma(1)=1, \Gamma(1-z)\Gamma(z)=\frac{\pi}{\sin(\pi z)} $$ we have $$ \Gamma(\frac{b}{2})\approx\frac{\pi}{\sin(\frac{b\pi}{2})}, \psi_0(1+\frac{b}{2})=-\gamma+\frac{\pi^2b}{12}+O(b^2).$$ From this, we have \begin{eqnarray} \psi_0(\frac{b}{2})&=&-\frac{2}{b}-\gamma+\frac{\pi^2b}{12}+O(b^2)\\ \psi_0(1+\frac{b}{2})&=&-\gamma+\frac{\pi^2b}{12}+O(b^2)\\ \psi_1(1+\frac{b}{2})&=&\frac{\pi^2}{6}+\frac{b}{2}\psi_2(1)+\frac{\pi^4b^2}{120}+O(b^3)\\ \psi_2(1+\frac{b}{2})&=&\psi_2(1)+\frac{\pi^4b}{30}+O(b^2), \end{eqnarray} and hence \begin{eqnarray} \frac{\partial^3 J(0,b)}{\partial a^2\partial b}&\approx&\frac{b\pi^3}{11520\sin(\frac{b\pi}{2})}(-2\pi^2+b^2\pi^4+60b\psi_2(1)). \end{eqnarray} So $$ \frac{\partial^3 J(0,0)}{\partial a^2\partial b}=\lim_{b\to0}\frac{\partial^3 J(0,b)}{\partial a^2\partial b}=\lim_{b\to0}\frac{b\pi^3}{11520\sin(\frac{b\pi}{2})}(-2\pi^2+b^2\pi^4+60b\psi_2(1))=-\frac{\pi^4}{2880}.$$
We will prove that $$I=-\frac{\pi^4}{2880}.$$ Indeed, let $$ J=\int_0^{\pi/2}\log^2(\sin x)\log(\cos x)\tan x \,dx $$ It is easy to see that $$\eqalign{J&=\int_0^{\pi/4}\log^2(\sin x)\log(\cos x)\tan x \,dx+ \int_{\pi/4}^{\pi/2}\log^2(\sin x)\log(\cos x)\tan x \,dx\cr &=\int_0^{\pi/4}\log^2(\sin x)\log(\cos x)\tan x \,dx+ \int_{0}^{\pi/4}\log^2(\cos x)\log(\sin x)\cot x \,dx\cr &=I }$$ Now, to calculate $J$ we make the substitution $t\leftarrow\sin^2x$: $$ J=\frac{1}{16}\int_0^1\frac{\log(1-u)}{1-u}\log^2(u)\,du $$ But $$\frac{\log(1-u)}{1-u}=-\left(\sum_{n=0}^\infty u^n\right)\left(\sum_{n=1}^\infty \frac{u^n}{n}\right) =-\sum_{n=1}^\infty H_nu^n $$ where $H_n=\sum_{k=1}^n1/k$. Hence $$J=-\frac{1}{16}\sum_{n=1}^\infty H_n\int_0^1u^n\log^2(u)du =-\frac{1}{8}\sum_{n=1}^\infty\frac{ H_n}{(n+1)^3} $$ Finally, since $H_{n}=H_{n+1}-\frac{1}{n+1}$, we get $$J=\frac{1}{8}\zeta(4)-\frac{1}{8}\sum_{n=1}^\infty\frac{H_n}{n^3}\tag{1}$$
The sum $\sum_{n=1}^\infty\frac{H_n}{n^3}$ is known, it can be evaluated as follows, first we have $$ H_n=\sum_{k=1}^\infty\left(\frac{1}{k}-\frac{1}{k+n}\right)= \sum_{k=1}^\infty \frac{n}{k(k+n)} $$ Thus $$ \sum_{n=1}^\infty\frac{H_n}{n^3}=\sum_{k,n\geq1}\frac{1}{n^2k(n+k)} =\sum_{k,n\geq1}\frac{1}{k^2n(n+k)} $$ Taking the half sum we find $$ \sum_{n=1}^\infty\frac{H_n}{n^3}=\frac{1}{2}\sum_{k,n\geq1}\frac{1}{kn(k+n)}\left(\frac{1}{k}+\frac{1}{n}\right)= \frac{1}{2}\sum_{k,n\geq1}\frac{1}{k^2n^2}=\frac{1}{2}\zeta^2(2) $$
Replacing in $(1)$ we obtain $$J=\frac{1}{8}\zeta(4)-\frac{1}{16}\zeta^2(2)=-\frac{\pi^4}{2880}.$$ as announced.$\qquad\square$