The continuous random variables $X$ and $Y$ have joint density function $f_{X,Y}(x,y)=1$ for $0<x<1$ and $2x<y<2$ and $0$ otherwise. Find $\operatorname{cov}(X,Y)$ and $\operatorname E(X\mid Y=y)$
So I have $\operatorname{cov}(X,Y)=\operatorname E[(X-\mu_X)(Y-\mu_Y)]$
I computed $f_X(x)=\int_0^2 1\,dy=2$
$$f_Y(y)=\int_0^{y/2} 1\,dx=\frac{y}{2}$$
Trying to compute $\mu_X=\int_0^{y/2} xf_X(x)\,dx=\int_0^2 2x\,dx=4$
and then $\mu_Y(y)=\int_0^2 y\frac{y}{2} \, dy=\frac{y^3}{6} \vert^2_0 = \frac{4}{3}$
So I have $E[(X-4)(Y-\frac{4}{3})]$
Trying to use $E(g(X,Y))=\int\int g(s,t)f_{X,Y}(s,t)\,ds\,dt$
$$\int_0^{\frac{y}{2}}\int_0^2(x-4)(y-\frac{4}{3})1\,dy\,dx$$
$$\int_0^{\frac{y}{2}}(x-4)(\frac{1}{2}y^2-\frac{4}{3}y\vert_0^2)$$
$$=\int_0^{\frac{y}{2}}(x-4)(2-\frac{8}{3}) \, dx$$
$$\int_0^{\frac{y}{2}} 2x-\frac{8}{3}x-8+\frac{32}{3} \, dx$$
$$=\frac{y^4}{2}-\frac{2y^4}{3}-4y^2+\frac{16}{3}y^2$$
which is not correct.
First we have to calculate $f_Y(y)$. It does not depend on $x$.
We have $2x<y\Rightarrow x<\frac12y$. Thus
$$f_Y(y)=\int_{0}^{\frac12y} f_{X,Y}(x,y) \, dx=\int_{0}^{\frac12y} 1 \, dx$$
And therefore $\mu_y=\int\limits_0^2 y\cdot f_Y(y) \, dy=\int\limits_0^2 y\cdot \left(\int\limits_{0}^{\frac12y} 1 \, dx \right) \, dy$
Next you calculate the covariance. You´re on the right track.
We have $\mu_x=\int\limits_0^1 x\cdot (2-2x) \, dx=\frac13$ and $\mu_y=\frac43$
$$Cov(X,Y)=\int_0^{1}\int_{2x}^2 \left(x-\frac13\right)\cdot \left(y-\frac{4}{3}\right) \, 1 \, dy \,dx$$
No variable at the bounds of the outer integral. Again: Covariance is a number.