Suppose $X$, $Y$ are random variables with joint density: $$f_{X,Y}(x,y)=\frac{e^{-y/2}}{2\pi\sqrt{x(y-x)}}$$ where $0< x< y$.
(a) Find the distribution of $Y$. Hint: for integration use the substitution $x=ys$
(b) Compute $E(X|Y=1)$
I'm not sure how to use the joint density to find the ditribution of $Y$. And I have no idea how to compute that expectation.
On
To find the marginal density from a joint density function, you integrate out the unnecessary variable.
So for example, to find $f_Y(y)$, you integrate $\int_0^y{f(x,y)dx}.$
From here, you know $f_{X|Y}(x|y)=\frac{f_{X,Y}(x,y)}{f_Y(y)}$
Now what is $E[X|Y=y]$? Isn't it just $\int{x\cdot f(x|y=1)dx}$
An easy approach is to interpret probabilistically the change of variable suggested as a hint, that is, to compute the distribution of $(Z,Y)$, where $X=YZ$. One should find $$ f_{Z,Y}(z,y)=\frac1{\pi\sqrt{z(1-z)}}\mathbf 1_{0\lt z\lt1}\cdot\frac12\mathrm e^{-y/2}\mathbf 1_{y\geqslant0}. $$ Thus, $Z$ and $Y$ are independent with respective densities $f_Z$ and $f_Y$ defined by $$ f_{Z}(z)=\frac1{\pi\sqrt{z(1-z)}}\mathbf 1_{0\lt z\lt1},\qquad f_Y(y)=\frac12\mathrm e^{-y/2}\mathbf 1_{y\geqslant0}. $$ In passing, this answers (a). Note that $Y$ is exponential with parameter $\frac12$ and that $Z$ follows the arcsine distribution.
To solve (b), note that $E[X\mid Y=1]=E[YZ\mid Y=1]=E[1\cdot Z\mid Y=1]=E[Z]$ since $Z$ is independent of $Y$. Furthermore, the symmetry of $f_Z$ by the transformation $z\to1-z$ implies that $E[Z]=E[1-Z]$ hence $E[Z]=\frac12$ and $E[X\mid Y=1]=\frac12$.
Note that one can also compute directly $E[Z]$ using the explicit form of $f_Z$, naturally, and that the change of variable $z=\sin^2t$ works like a charm here, which should come as no surprise since the identity $Z=\sin^2T$ with $T$ in $[0,\frac\pi2]$ defines a random variable $T$ uniform on $[0,\frac\pi2]$, as the name "arcsine" of the distribution of $Z$ suggests.