How may one demonstrate this integral is equal to this result?
$$\int_{0}^{1}\arccos\sqrt{1-x}\ln(x-x^2)\ln\left(\frac{1}{x}-1\right)\mathrm dx=\pi\zeta(2)-2\pi[1-\ln^2(2)]$$
Where do I start from?
How may one demonstrate this integral is equal to this result?
$$\int_{0}^{1}\arccos\sqrt{1-x}\ln(x-x^2)\ln\left(\frac{1}{x}-1\right)\mathrm dx=\pi\zeta(2)-2\pi[1-\ln^2(2)]$$
Where do I start from?
On
Just some thoughts, for the moment. Perhaps I'll add details later.
First of all, using the properties of logarithms you will easily find that your integral can be rewritten as
$$\int_0^1 \arccos\sqrt{1-x}\left(\ln^2(1-x) - \ln^2(x)\right)\ \text{d}x$$
A suitable change of variable like $1-x = z^2$ or some exponential manipulations shall work.
In any case you might also split the two integrals.
For the records, a very standard integral can be noticed right now:
$$\int_0^1 z \arccos(z) \ln^2(z)\ \text{d}z = \frac{1}{96} \pi \left(\pi ^2-6+3 \ln ^2(4)\right)$$
Other manipulations may arise from Taylor series, but this is all to be understood properly.
On
The variable change and the integration by parts reveal together that
$$I= 8\underbrace{\int_0^{\pi/2}\sin^2(x)\log^2(\sin(x))\textrm{d}x}_{\displaystyle \text{Beta function}}-8\underbrace{\int_0^{\pi/2}\sin^2(x)\log(\sin(x))\textrm{d}x}_{\displaystyle \text{Beta function}}$$ $$=\pi \zeta(2)-2\pi (1-\log^2(2)).$$
Extra information: see eq. (14) here http://mathworld.wolfram.com/BetaFunction.html
This answer will primarily show how Waiting most likely got his answer. To start off, let's first define $I$ to be the integral under question, and using the basic logarithmic rules we all learned in first grade, $I$ can be transformed as
$$\begin{align*}I & =\int\limits_0^1dx\,\arccos\sqrt{1-x}\log^2(1-x)-\int\limits_0^1dx\,\arccos\sqrt{1-x}\log^2x\\ & =\int\limits_0^1dx\,\arccos\sqrt x\log^2x-\int\limits_0^1dx\,\arccos\sqrt x\log^2(1-x)\tag1\end{align*}$$
From ($1$), make the substitution $x\mapsto\sqrt x$ to get rid of the radical in the $\arccos(\cdot)$ function. Therefore, ($1$) turns into the form
$$\begin{align*}I & =8\int\limits_0^1dx\, x\arccos x\log^2x-2\int\limits_0^1dx\, x\arccos x\log^2(1-x^2)\tag2\end{align*}$$
Next, substitute $x\mapsto\cos x$ to get
$$I=8\int\limits_0^{\pi/2}dx\, x\sin x\cos x\log^2\cos x-8\int\limits_0^{\pi/2}dx\, x\sin x\cos x\log^2\sin x\tag3$$
Now, integration by parts, as difficult as it may seem, can be used to reduce the integral into something the beta function can easily manage. First, denote $I_1$ and $I_2$ as the first and second integral respectively of ($3$). Letting $u=x$ and integrating by parts on $I_1$ gives
$$\begin{align*}I_1 & =\frac {\pi}2+4\int\limits_0^{\pi/2}dx\,\cos^2x\log^2\cos x-4\int\limits_0^{\pi/2}dx\,\cos^2x\log\cos x\\ & =\frac {\pi}2+4\int\limits_0^{\pi/2}dx\,\sin^2\log^2\sin x-4\int\limits_0^{\pi/2}dx\,\sin^2x\log\sin x\\I_2 & =\frac {\pi}2-4\int\limits_0^{\pi/2}dx\,\sin^2x\log^2\sin x+4\int\limits_0^{\pi/2}dx\,\sin^2 x\log\sin x\end{align*}$$
Note that in the second line, we exploited the identity $\cos\left(\frac {\pi}2-x\right)=\sin x$ and used the limit identity for integrals. Putting everything together in ($3$) leaves
$$\begin{align*}I & =8\int\limits_0^{\pi/2}dx\,\sin^2\log^2\sin x-8\int\limits_0^{\pi/2}dx\,\sin^2x\log\sin x\\ & =8\frac {\partial^2}{\partial a^2}\int\limits_0^{\pi/2}dx\,\sin^{a+2}x-8\frac {\partial}{\partial b}\int\limits_0^{\pi/2}dx\,\sin^{b+2}x\end{align*}$$
Using equation ($14$) from here gives the answer as
$$\begin{align*}I & =4\sqrt{\pi}\frac {\partial^2}{\partial a^2}\frac {\Gamma\left(\frac a2+\frac 32\right)}{\Gamma\left(\frac a2+2\right)}-4\sqrt{\pi}\frac {\partial}{\partial b}\frac {\Gamma\left(\frac b2+\frac 32\right)}{\Gamma\left(\frac b2+2\right)}\\ & =\frac {\pi}6\biggr[\pi^2+6\log 2\left(\log 4-2\right)-6\biggr]-\pi+2\pi\log 2\end{align*}$$
A little simplifying reveals the answer
$$\int\limits_0^1dx\,\arccos\sqrt{1-x}\log(x-x^2)\log\left(\frac 1x-1\right)\color{blue}{=\frac {\pi^3}6+2\pi\log^22-2\pi}$$