given $(X,Y)$ random variable 2D continuously with the density function $$f_{X,Y}(x,y)=\begin{cases}\frac{1}{{30}}\cdot x^{m}\cdot ye^{-y} & x \in [0,y] \ , \ y>0 \\ 0 & \textrm{otherwise} \end{cases}$$
I need to find the value of $m$
I know I should use the properties of the density function $$1=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}f(x,y)dxdy$$ $$1=\int_{0}^{y}\int_{-\infty}^{\infty}\frac{1}{{30}}\cdot x^{m}\cdot ye^{-y}dxdy$$
I dont understand how to determine the limits of $x$ from the given about it.
In the computation of $\displaystyle\iint_{\Bbb R^2}f(x,y)\; dx\; dy$ you can use Fubini and
The two ways are: $$ \begin{aligned} 1 &= \int_{y\in[0,\infty)}dy\int_{x\in[0,y]} \frac 1{30}x^m\; y\; e^{-y}\; dx \\ &= \int_{y\in[0,\infty)}dy\cdot \frac 1{30}\cdot\frac {y^{m+1}}{m+1}y\; e^{-y}\\ &= \frac 1{30(m+1)}\cdot \Gamma(m+3) \qquad\text{ and if $m$ is integer} \\ &= \frac 1{30(m+1)}\cdot (m+2)! \\ &=\frac 1{30}(m+2)\cdot m! \ , \\[4mm] 1 &= \int_{x\in[0,\infty)}dx\int_{y\in[x,\infty)} \frac 1{30}x^m\; y\; e^{-y}\; dy \\ &= \int_{x\in[0,\infty)}dx\cdot \frac 1{30}x^m\; (x+1)\; e^{-x} \\ &=\frac 1{30}\cdot(\Gamma(m+2)+\Gamma(m+1)) \qquad\text{ and if $m$ is integer} \\ &= \frac 1{30}\cdot ((m+1)!+m!) \\ &=\frac 1{30}\cdot m!\cdot((m+1)+1) \\ &=\frac 1{30}(m+2)\cdot m! \ . \end{aligned} $$ For $m=3$ we have indeed equality.
The equation to be solved $(m+2)\Gamma(m+1)=30$ has a unique solution $m>0$ since the L.H.S. is an increasing function of $m$.
But note that the $\Gamma$ function comes with "its own caprice", we find for instance...