The joint density function of two random variables $X$ and $Y$ is
$$f_{X,Y}(x,y)=\begin{cases}0.0009x(1+y), \quad\text{if $(x,y)\in \Omega$}\\ 0, \quad\text{else}\end{cases}$$ and the marginal distributions are $$f_X(x)=0.0036(21x-2x^2-x\sqrt{x}),\quad\text{and}\quad f_Y(y)=0.0009(1+y)\frac{y^4}{512}$$ where $$\Omega\{(x,y):4\sqrt{x}<y \quad\text{and}\quad 0<y<12\}$$ I want to calculate the following conditional probability $$P(X<4|Y>4)$$ My idea was to use the following definition
If $X$,$Y$ are continuous random variables such that $X\in A$ and $Y\in B$, then the conditional probability is defind as $$P(X\in A | Y\in B)=\frac{\int_B P(X\in A|y)f_Y(y)dy}{\int_Bf_Y(y)dy}$$ in my case this is translated to $$P(X<4|Y>4)=\frac{\int_{4}^{12} P(X<4|y)f_Y(y)dy}{\int_{4}^{12}f_Y(y)dy}$$ But I struggle to calculate the numerator $\int_{4}^{12} P(X<4|y)f_Y(y)dy$ because of the integration limits. Can anybody help me to solve this simple problem?
Keeping the point on joint density function that I made in comments aside, to find $\small P(X \lt 4 | Y \gt 4)$, you can either work with marginal density or directly with the joint density function.
If you look at the diagram, the shaded region together (marked $1$ and $2$) will give you $ \small P(Y \gt 4)$. Also the region shaded in light blue (marked $1$) will give us $\small P((X \lt 4) \cap (Y \gt 4))$.
Given the region shaded in light green is $ \small P(X \gt 4)$, $\small P((X \lt 4) \cap (Y \gt 4)) = P(Y \gt 4) - P(X \gt 4)$
So, $\displaystyle \small P(Y \gt 4) = \int_4^{12} \int_0^{y^2/16} f(x,y) \ dx \ dy \ \ $ ...($i$)
$\displaystyle \small P((X \lt 4) \cap (Y \gt 4)) = P(Y \gt 4) - \int_8^{12} \int_4^{y^2/16} f(x,y) \ dx \ dy \ \ $ ...($ii$)
dividing ($ii$) by ($i$) will give us $ \small P(X \lt 4 | Y \gt 4)$