$$\int\arcsin\left(\sin^{2}x\right)dx$$
I am not able to find a closed form elementary solution for this, though I have no reason to believe it exists.
But trying out the Definite Integral as follows:
$$I=\int_{0}^{\frac{\pi}{2}}\arcsin\left(\sin^{2}x\right)dx$$
Using the Series Expansion for $\arcsin(x)$: $$\arcsin(x)=\sum_{n=0}^{\infty}\frac{\left(2n\right)!}{2^{2n}\left(n!\right)^{2}}\ \frac{x^{2n+1}}{2n+1}$$ $$\arcsin\left(\sin^{2}x\right)=\sum_{n=0}^{\infty}\frac{\left(2n\right)!}{2^{2n}\left(n!\right)^{2}}\ \frac{\left(\sin x\right)^{4n+2}}{2n+1}$$ $$I=\sum_{n=0}^{\infty}\frac{\left(2n\right)!}{2^{2n}\left(n!\right)^{2}\left(2n+1\right)}\ \int_{0}^{\frac{\pi}{2}}\left(\sin x\right)^{4n+2}dx$$
We are aware of the closed form of the Integral: $$\int_{0}^{\frac{\pi}{2}}\left(\sin x\right)^{a}dx=\frac{\sqrt \pi}{2}\frac{\Gamma\left(\frac{a+1}{2}\right)}{\Gamma\left(\frac{a}{2}+1\right)}$$
$$I=\frac{\sqrt \pi}{2}\sum_{n=0}^{\infty}\frac{\left(2n\right)!}{2^{2n}\left(n!\right)^{2}\left(2n+1\right)}\ \frac{\Gamma\left(\frac{4n+3}{2}\right)}{(2n+1)!}$$ $$I=\frac{\sqrt{\pi}}{2}\sum_{n=0}^{\infty}\frac{\Gamma\left(\frac{4n+3}{2}\right)}{2^{2n}\left(n!\right)^{2}\left(2n+1\right)^{2}}\ $$
Wolfram Gives a Closed Form for the Summation in terms of Hypergeometric Function as follows: $$\sum_{n=0}^{\infty}\frac{\Gamma\left(\frac{4n+3}{2}\right)}{2^{2n}\left(n!\right)^{2}\left(2n+1\right)^{2}}\ =\frac{\sqrt{\pi}}{2}\,_{4}F_{3}\left(\frac{1}{2},\frac{1}{2},\frac{3}{4},\frac{5}{4};1,\frac{3}{2},\frac{3}{2};1\right)$$ Hence, $$I=\frac{\pi}{4}\,_{4}F_{3}\left(\frac{1}{2},\frac{1}{2},\frac{3}{4},\frac{5}{4};1,\frac{3}{2},\frac{3}{2};1\right)$$
I have seen some cases where these hypergeometric functions end up in terms of closed form expressions.
Well my question is:
- Whether the Indefinite Integral has a solution.
- Whether the Definite Integral has a Closed Form Solution.
- Whether the Hypergeometric Series has a Closed Form, which will lead us directly to the Definite Integral.
The answer to your second question is YES:
$$\begin{align*} \int_0^{\tfrac\pi2} \arcsin\left(\sin^2x\right) \, dx &= \int_0^1 \frac{\arcsin y^2}{\sqrt{1-y^2}} \, dy \tag1 \\ &= \int_0^1 \left[\int_0^{\sqrt y} \frac{2yz}{\sqrt{1-y^2z^4}}\right] \, \frac{dy}{\sqrt{1-y^2}} \tag2 \\ &= \int_0^1 \left[\int_{z^2}^1 \frac{2y}{\sqrt{1-y^2}\sqrt{1-z^4y^2}} \, dy\right] \, z \, dz \\ &= 2 \int_0^1 \underbrace{\ln \left(z^2 + \sqrt{1+z^4}\right)}_{\equiv \operatorname{arsinh}z^2} \, \frac{dz}z \tag3\\ &= -4 \int_0^1 \frac{z \ln z}{\sqrt{1+z^4}} \, dz \tag4\\ &= - \int_0^1 \frac{\ln w}{\sqrt{1+w^2}} \, dw \tag5 \end{align*}$$
The last expression can be evaluated using this antiderivative, giving a closed form of
$$\boxed{\ln2\ln\left(1+\sqrt2\right)+\frac12\ln^2\left(1+\sqrt2\right) - \frac{\pi^2}{12} + \frac12 \Re \operatorname{Li}_2\left(3+2\sqrt2\right)}$$