Suppose that $(X,Y)$ is uniformly distributed on the subset of $\; \mathbb R^2$ defined by the inequalities $0 < X < 1$ and $0 < Y < X^2$.
Determine the probability density function of the random variable $Z = XY$
The $X^2$ in the $Y$ interval is seriously throwing me off and I'm not sure where to start. I have
$$f_{X,Y}(x,y) = 3 \qquad (x,y)\ \epsilon\;(0,1)\times(0,X^2)$$ $$f_{X,Y}(x,y) = 0 \qquad \qquad \qquad \quad otherwise$$
But I do not know where to go from here. I need to be able to explain my answer so I'd like to understand how/why it works. Thanks!
You have found that$f_{X,Y}(x,y)=3\cdot\mathbf 1_{0<x<1, 0<y<x^2}$ . $\checkmark$
From there use the Jacobian Transformation Theorem.
$$\begin{align}f_{X,Z}(x,z) &= \begin{Vmatrix}\dfrac{\partial[x,z/x]}{\partial[x,z]}\end{Vmatrix}~f_{X,Y}(x,z/x)\\[1ex] &= \dfrac 1{\lvert x\rvert}\cdot 3\cdot\mathbf 1_{0< x< 1~,~ 0< z/x< x^2}\\[1ex] &= \dfrac 3{x}\cdot\mathbf 1_{0< z< 1~,~ \sqrt[3]z< x< 1}\end{align}$$
After which integration with respect to $x$ finds the marginal distribution. (Notice the support)