I have the following problem and solution:
I don't understand how the bounds of integration were changed from 0 to 1, to $x^2$ to 1. I see where $1/\sqrt{y}$ was substituted in for $f(x|y)$ and 1 was substituted in for $g(y)$, but I don't see any justification for changing the bounds.
The limits of integration are changed because outside of those values the density is equal to zero.
The random variable X is uniform on $(0, \sqrt{y})$ which means that outside of this range the density function is zero. That is the conditional density is,
$ f(x \vert y) = \left\{ \begin{array}{lr} \frac{1}{\sqrt{y}} & : 0 < x < \sqrt{y} , \\ 0 & : \text{otherwise .} \end{array} \right. $
$ 0 < x < \sqrt{y}$ in terms of y is $ x^2 < y $. The conditional density $f(x \vert y )$ is zero outside of this range and hence will not contribute to the value of the integral. The upper limit is still $1$ because $Y \sim Unif(0,1)$. Therefore,
$ f(x) = \int_0^1 f(x \vert y) g(y) \mathrm{d}y = \int_0^{x^2} f(x \vert y) g(y) \mathrm{d}y + \int_{x^2}^1 f ( x \vert y) g(y) \mathrm{d}y = \int_{x^2}^1 f ( x \vert y) g(y) \mathrm{d}y . $