Suppose we have to find the integral of the function $\sqrt{\frac{x}{1-x}}$. To solve this, I set $\sqrt{x} = -\sin\theta$, where theta belongs to [0, -π/2) which implies that $x = \sin^2\theta$. The differential $dx$ can then be expressed as $dx = 2\sin\theta\cos\theta d\theta$.So
$$\int \sqrt{\frac{x}{1-x}} \, dx$$
$$= \int \sqrt{\frac{sin² \theta}{1-sin² \theta}} \, 2\sin\theta\cos\theta d\theta$$ $$= \int \sqrt{\frac{sin² \theta}{cos² \theta}} \, 2\sin\theta\cos\theta d\theta$$ $$= \int 2sin² \theta d\theta $$
$$= \theta - \frac{\sin 2\theta}{2} + C $$ $$=-\arcsin(\sqrt{x}) - \sqrt{x(1 - x)} + C$$ But the correct answer is $$\arcsin(\sqrt{x}) - \sqrt{x(1 - x)} + C$$ What am I doing wrong?
$$ \sqrt{\frac{\sin^2 \theta}{\cos^2 \theta}} = \left\lvert \frac{\sin \theta}{\cos \theta}\right\rvert = - \frac{\sin \theta}{\cos \theta} $$
since $\sin\theta \leq 0$ and $\cos\theta > 0$. So your last integrand has the wrong sign.
You made a second sign error, since $\sin(2\theta) = -\sqrt{x(1-x)}$ but you substituted $\sqrt{x(1-x)}$. This partly canceled your first sign error, which is why the entire answer doesn't have its sign flipped, only the first term.
Why would you want to work in the fourth quadrant instead of the first? It's much easier to keep track of signs when $\lvert\sin\theta\rvert = \sin\theta$ rather than $-\sin\theta$.