Let $f(x)=(\sin(\tan^{-1}x)+\sin(\cot^{-1}x))^2-1, |x|>1$. If $\frac{dy}{dx}=\frac12\frac d{dx}(\sin^{-1}(f(x)))$ and $y(\sqrt3)=\frac{\pi}{6}$, then $y(-\sqrt3)=?$
$$f(x)=(\frac{x}{\sqrt{x^2+1}}+\frac{1}{\sqrt{x^2+1}})^2-1=\frac{2x}{1+x^2}$$ $$\frac{dy}{dx}=\frac12\frac d{dx}(\sin^{-1}(\sin(2\tan^{-1}x)))$$ $$y=\tan^{-1}x+c$$ Using, $y(\sqrt3)=\frac{\pi}{6}$, I get, $c=-\frac\pi6$. Thus, $y(-\sqrt3)=-\frac\pi2$. But the answer is given as $\frac{5\pi}6$.
$\sin(\cot^{-1}x)=\sin\left(\dfrac\pi2-\tan^{-1}x\right)=\cos(\tan^{-1}x)$
$$\implies f(x)=\left(\sin(\tan^{-1}x)+\sin(\cot^{-1}x)\right)^2-1=\sin2\left(\tan^{-1}x\right)$$
Now $\sin^{-1}\left(\sin(2\tan^{-1}x )\right)=\begin{cases} \pi-2\tan^{-1}x &\mbox{if } 2\tan^{-1}x>\dfrac\pi2\iff x>\tan\dfrac\pi4 \\ -\pi-2\tan^{-1}x& \mbox{if } 2\tan^{-1}x<-\dfrac\pi2 \end{cases}$