I have been wondering this since last night when I read about Bayesian inference. More precisely, the set up is like this.
$X$ takes values in $\mathcal{X}$ and suppose that $X$ has pdf $f(x;\theta)$, where $\theta$ itself is a random variable taking values in $\Omega$. To make it clear, I will call $Y$ the random variable whose realization is $\theta$. Suppose that $Y$ has pdf $\lambda(\theta)$. My question is, what is the joint distribution of $X$ and $Y$?
The explanation in the book, which is very logical and I understood it all, goes like this:
We consider $f(x;\theta)$ as the conditional pdf of $X$ given $Y=\theta$. Hence, the joint distribution of $X$ and $Y$ is $f(x,\theta)=f(x;\theta)\lambda(\theta)$.
But I can't help but wonder what the rigorous justification for this explanation was. I know that if $X$ and $Y$ has a joint pdf then I can find the conditional pdf of $X$ given $Y=\theta$ by dividing the joint pdf by the pdf of $Y$. Over here, I do not have the joint pdf to start with. I know how to construct the joint distribution of $X$ and $Y$ if they are not related in this particular way, but in this case, I am not sure.
There is one thing that pops up in my mind that I think maybe related to, and can justify, this question. This thing is about transition probability kernel and a way of constructing product measures that I read in a probability book a while ago. I will reread it but in the mean time, I will be appreciate if you have any thought about my question.
Thank you.
It appears that $f(x;\theta)$ is the conditional probability density function when given $\theta$ as the realisation of $Y$.
This is more usually written as something like $f_{X\mid Y}(x\mid \theta)$.
Likewise $\lambda(\theta)$ would be written as $f_Y(\theta)$, the marginal probability density function for $Y$.
So you'd have the joint probability density function : $$f_{X,Y}(x, \theta)= f_{X\mid Y}(x\mid \theta)~f_Y(\theta)\qquad\text{or}\qquad f_{X\mid Y}(x\mid \theta)=\dfrac{f_{X,Y}(x,\theta)}{f_Y(\theta)}$$
Where they have written: $$f(x,\theta) = f(x; \theta)~\lambda(\theta)\qquad\text{or}\qquad f(x;\theta)=\dfrac{f(x,\theta)}{\lambda(\theta)}$$
tl;dr It's just a notation difference.