Problem Statement:
$X$ and $Y$ are random variables with joint density:
$f_{XY}(x,y)=\left\{\begin{matrix} 8xy, & x \geq 0, y \geq 0, x^2 + y^2 \leq 1 \\ 0, & otherwise \end{matrix}\right.$
(a) Find the marginal density for $X$.
(b) Find the marginal distribution function for $X$.
(c) Find the conditional density function for $Y$ given $X = x$.
(d) Find the conditional distribution function for $Y$ given $X = x$.
(e) Find the joint distribution function for $X$ and $Y$ in the region $x \geq 0, y \geq 0, x^2 + y^2 \leq 1$.
(f) Find the joint distribution function for $X$ and $Y$ in the region $x \geq 1, 0 \leq y < 1$.
So far I have determined that the area in question is the positive quadrant of the unit circle:
Area of Interest: $x \geq 0, y \geq 0, x^2 + y^2 \leq 1.$
I'm having trouble determining the bounds of integration for the marginal density for $X$. My initial guess is that $y$ is moving from $0$ to $\sqrt{1 - x^2}$ and that may be the region of integration for each $x$.
Also, I'm not sure what the conditioning of the marginal density will be. My guess would be for $x \geq 0, x^2 \leq 1$, otherwise $0$.
Any help is much appreciated.
The marginal density of $X$ at $x$ is obtained by integrating the joint density $f_{X,Y}(x,y)$ over all $y$. Here that joint density is $0$ outside the quadrant of the circle, which you already have a picture of.
Assuming $0 \le x \le 1$, the vertical line at abscissa $x$ cuts this region in a line segment extending from $y=0$ (on the axis) to $y = \sqrt{1-x^2}$ (on the top half of the circle). So you are correct: you integrate from $y=0$ to $y=\sqrt{1-x^2}$. But if $x < 0$ or $x > 1$, there is nothing nonzero to integrate and you get marginal density $0$.