Suppose that X is a continuous random variable with density function f(x)=(1-x)20x^3 for [0,1] and 0 otherwise. Generate random variable Y by choosing a point uniformly and randomly from the interval [X,1]. Find the unconditional density of Y.
This means, find f(y). To do this, do I first say f(y|x)=1/(1-x)=f(x,y)/f(x)? Then I can find the joint density to be 20x^3. I do not know how to proceed from this point though.
Actually, the joint density $f$ you should find is defined by $$ f(x,y)=20\,x^3\,\mathbf 1_{0\lt x\lt y\lt1}. $$ According to my experience (on MSE and elsewhere), omitting the indicator function in the density can, and often does, cause severe problems of comprehension. Anyway, as you know, given the density of $(X,Y)$, the density $f_Y$ of $Y$ is such that $$ f_Y(y)=\int_\mathbb Rf(x,y)\,\mathrm dx=20\,\mathbf 1_{0\lt y\lt1}\int_0^yx^3\,\mathrm dx=5\,y^4\,\mathbf 1_{0\lt y\lt1}. $$ Sanity check: The function $f_Y$ should be nonnegative everywhere and its integral should be $1$. You might want to check these.