Let $\xi,\eta$ be independent random variables, both with uniform distribution on $[0,2]$. Find $E[\eta^2|\xi/\eta]$.
My attempt to solve the problem is in the attached file. I believe I solved it, but made a mistake on the way. Range of $u$ becomes $[0,\infty]$, so if I integrate joint probability $1/8$, I get $\infty$.
You cannot conclude that the range is $[0,\infty]$ directly. First look a t the hyperbolic region(sort of) and split them up.
Let $U=Y$ and $V=\frac{X}{Y}$
Then
$$|J|=\begin{vmatrix} 0 & 1\\\frac{1}{y} &\frac{-x}{y^{2}} \end{vmatrix}=\frac{-1}{y}$$
Then you have $f(u,v)=\frac{y}{4}=\frac{u}{4},\,0\leq u\leq 2, 0\leq uv\leq 2 $.
So $$f(v)=\begin{cases}\int_{0}^{\frac{2}{v}}\frac{u}{4}\,du\,, 1\leq v<\infty\\ \int_{0}^{2}\frac{u}{4}\,du,\,0\leq v<1\end{cases}$$
$$f(v)=\begin{cases} \frac{1}{2v^{2}}\,, 1\leq v<\infty\\ \frac{1}{2} \,,0\leq v <1\end{cases}$$.
So conditional expectation is:-
$$\mathbb{E}[Y^{2}|\frac{X}{Y}=v]=\begin{cases}2v^{2}\int_{0}^{\frac{2}{v}}u^{2}\frac{u}{4}\,du\,\,,1\leq v<\infty\\2\int_{0}^{2}u^{2}\frac{u}{4}\,du\,\,,0\leq v<1\end{cases}$$