Suppose that $(U,V)$ has the following joint pdf: $$f_{U,V}(u,v)=\exp(-\theta u-\theta^{-1}v)$$ , where $u\geq0$, $v\geq0$, $\theta>0$. Define $X=UV$ and $Y=U/V$. Find the marginal pdf of $X$ and $Y$.
With transformation, I got $U=\sqrt{XY}$ and $V=\sqrt{X/Y}$ and the absolute value of Jacob is $1/2Y$. Then, the joint pdf of $(X,Y)$ is $$f_{X,Y}(x,y)=\exp(-\theta \sqrt{XY}-\theta^{-1}\sqrt{X/Y})1/2Y, X,Y>0.$$
But I am not sure how to get the marginal pdf?
Thank you!
As you had found, pdf of $(X,Y)$ is
$$f_{X,Y}(x,y)=\frac{1}{2y}\exp\left[-\left(\theta\sqrt{xy}+\frac{1}{\theta}\sqrt{\frac{x}{y}}\right)\right]\,\mathbf1_{x>0,y>0}$$
So pdf of $Y$ is
\begin{align} f_Y(y)&=\frac{1}{2y}\int_0^\infty \exp\left[-\left(\frac{\theta^2y+1}{\theta\sqrt{y}}\right)\sqrt{x}\right]\,dx\,\mathbf1_{y>0} \\&=\frac{1}{y}\int_0^\infty te^{-\alpha t}\,dt\,\mathbf1_{y>0} \tag{*} \\&=\frac{1}{y\alpha^2}\mathbf1_{y>0} \end{align}
In $(*)$ we substituted $t^2=x$ and $\alpha=\frac{\theta^2y+1}{\theta\sqrt{y}}(>0)$.
That is, $$f_Y(y)=\left(\frac{\theta}{\theta^2y+1}\right)^2\mathbf1_{y>0}$$
Indeed since $U$ and $V$ are independent exponential variables, it can be verified that $\theta^2Y\sim F_{2,2}$, an F distribution with $(2,2)$ degrees of freedom.
And substituting $u^2=y$, density of $X$ is given by
$$f_X(x)=\int_0^\infty \frac{1}{u}\exp\left[-\left(\theta\sqrt{x}u+\frac{\sqrt x}{\theta u}\right)\right]\,du\,\mathbf1_{x>0}$$
The above can be rewritten as $$f_X(x)=2K_0(2\sqrt{x})\,\mathbf1_{x>0}\,,$$
where $K_0(\cdot)$ is a Modified Bessel function of the second kind.
Notably the distribution of $X$ is independent of $\theta$.