So I think I know how to solve (a) correctly for this problem, but I keep getting answers to (b) that don't integrate to be $1$. I think (c) follows straightforwardly from there so (b) is the big question.
Consider a disk of radius $R : \{B={ (x,y) : x^2+y^2 \leq R^2\} }$. A point $P= (X,Y)$ is chosen at random with uniform distribution in $B$. Let also $D$ be the distance of $P$ from the origin.
(a) Find the marginal densities of $X$ and $Y$.
(b) Find the pdf of $D$.
(c) Find $E(D)$, the expectation of $D$.
I think the marginal for $X$ is $f(x)=\dfrac{2\sqrt{R^2-x^2}}{\pi R^2}$ and $Y$ is $f(y)=\dfrac{2\sqrt{R^2-y^2}}{\pi R^2}$. I tried to find the density of $D=\sqrt{X^2+Y^2}$ by $f(d)=\sqrt{{f(x)}^2+{f(y)}^2}$ but it hasn't been a valid density so far.
Here's a simple approach in which you forget about $X$ and $Y$. Take a disk of radius $d\leq R$. Since the points are uniformly distributed, the probability that a point will fall within the disk $B_d = \{(x,y):x^2+y^2\leq d^2\}$ is just the area of $B_d$ divided by the area of the whole disk:
$$ P((X,Y)\in B_d) = \frac{d^2}{R^2} $$ Then the PDF is simply: $$ \frac{\mathrm{d}P}{\mathrm{d}D} = \frac{2D}{R^2} $$
An alternative would be to do a change of variables from $(x,y)$ to $(r,\phi)$ (polar coordinates). Then $(x,y)(r,\phi)=(r\cos\phi, r\sin\phi)$. Change of variables then says that the PDF of $(r,\phi)$ is: $$ g(r,\phi)=f(x(r,\phi),y(r,\phi))\,\,\left\vert\mathrm{det}\!\left(\frac{\partial(x,y)}{\partial(r,\phi)}\right)\right\vert $$ In our case $f(x(r,\phi),y(r,\phi)) = 1/\pi R^2$. The last part on the right-hand side is the absolute value of the determinant of the Jacobian matrix:$$ \mathrm{det}\!\left(\frac{\partial(x,y)}{\partial(r,\phi)}\right)= \begin{vmatrix} \frac{\partial x}{\partial r} & \frac{\partial x}{\partial\phi} \\ \frac{\partial y}{\partial r} & \frac{\partial y}{\partial\phi}\notag \end{vmatrix}= \frac{\partial x}{\partial r}\frac{\partial y}{\partial\phi}-\frac{\partial x}{\partial\phi}\frac{\partial y}{\partial r}=r $$ So it follows that $g(r,\phi)=r/\pi R^2$. We get the marginal probability for $r$ as: $$ \frac{\mathrm{d}P}{\mathrm{d}r} = \int_0^{2\pi}r/\pi R^2\mathrm{d}\phi=\frac{2r}{R^2} $$ This agrees with our previous result. There are some technicalities here: going from cartesian to polar coordinates has a discontinuity around the positive $x$-axis as $\phi$ jumps there, so strictly speaking there are problems with changing variables. I don't know enough theory to provide a fully rigorous justification, but the rule of thumb is: the discontinuous part is only on the positive $x$-axis, which has no surface area ("has measure zero"), so it "can't spoil anything", so to speak. Maybe someone else can explain precisely why the jump in $\phi$ is irrelevant.