A generalization of the beta distribution is the Dirichlet distribution. In its bi-variate version, (X,Y) have pdf
$f(x,y) = Cx^{a-1}y^{b-1}(1-x-y)^{c-1}, 0<x<1, 0<y<1, 0<y<1-x<1$,
where $a>0,b>0, c> 0 $ are constants.
(a)Show that $C = \frac{\Gamma(a)\Gamma(b)\Gamma(c)}{\Gamma(a+b+c)}$
(b) Show that, marginally, both X and Y are beta.
(c) Find the conditional distribution of $Y|X=x$ and show that $Y/(1-x)$ is beta(b,c).
(d) Show that $E(XY) =\frac{ab}{(a+b+c+1)(a+b+c)}$, and find their covariance.
Attempt at (a):
$\int_0^1\int_0^{1-x} x^{a-1}y^{b-1}(1-x-y)^{c-1}dx dy$
$\int_0^1x^{a-1}\int_0^{1-x} y^{b-1}(1-x-y)^{c-1}dx dy$
Let $u=\frac{y}{(1-x)}$
$\int_0^1x^{a-1}(1-x)^{b+c-1}dx\int_0^1 u^{b-1}(1-u)^{c-1}du$
$\int_0^1x^{a-1}(1-x)^{b+c-1}dx*\frac{\Gamma(b)\Gamma(c)}{\Gamma(b+c)}$
$\frac{\Gamma(b)\Gamma(c)}{\Gamma(b+c)}\int_0^1x^{a-1}(1-x)^{b+c-1}dx$
$\frac{\Gamma(b)\Gamma(c)}{\Gamma(b+c)}\frac{\Gamma(a)\Gamma(b+c)}{\Gamma(a+b+c)}$
$\frac{\Gamma(a)\Gamma(b)\Gamma(c)}{\Gamma(a+b+c)}$
Not sure how to correctly do parts b-d
For part (b), you already did it when you did part (a), since the marginal distribution of $X$ is given by $$f_X(x) = \int_{y=0}^{1-x} f_{X,Y}(x,y) \, dy,$$ which you evaluated already.
For part (c), the conditional distribution is given by $$f_{Y\mid X}(y) = \frac{f_{X,Y}(x,y)}{f_X(x)}.$$
For part (d), you would calculate $$\mathrm{E}[XY] = \int_{x=0}^1 \int_{y=0}^{1-x} xy f_{X,Y}(x,y) \, dy \, dx.$$