I was studying the concept of sufficient statistics, and I saw a few documents (such as this one https://ocw.mit.edu/courses/economics/14-381-statistical-method-in-economics-fall-2013/lecture-notes/MIT14_381F13_lec4.pdf) where people consider conditional probability densities of the form $f_{X|T=t}(x)$ or joint densities of the form $f_{X,T}(x,t)$, where $X$ and $T$ are real random variables and $T = T(X)$ is a function of $X$.
I was surprised reading this kind of thing, since the vector $(X, T)$ does not admit a joint density, and in this case we can't even define the conditional density $f_{X|T=t}(x)$ (can we?).
So I started to think about the following problem:
Let $X, Y$ be two real random variables such that $Y = h(X)$. We consider that $X$ admits the density $f_X(x)$. How can we define the image measure $\mu_{X,Y}$ of the couple $(X, Y)$, such that, for $A \in \mathcal{B}({\mathbb{R}^2})$:
$$ P((X, Y) \in A) = \int_A \mu_{X,Y}(\mathrm{d}x, \mathrm{d}y) $$
My first try was to define the set $C = \{ (x,y) \in \mathbb{R}^2: y = h(x)\}$, the projection $\pi_x: (x,y)\mapsto x$ and then:
\begin{equation} \label{eq:1}\tag{1} P((X, Y) \in A) = \int_{\pi_x(A\cap C)} f_X(x) \;\mathrm{d}x \end{equation}
But I still couldn't write it as a measure on $\mathbb{R}^2$. Then I found the answer of spaceisdarkgreen to this question Joint density of dependent random variables, where he defines:
\begin{equation} \label{eq:2}\tag{2} f_{X, Y}(x,y) = f_X(x)\delta(y - h(x)) \end{equation}
I understood the idea behind this definition, but does it formally define a measure on $\mathbb{R}^2$? Or the Dirac should be interpreted as a distribution and not a measure? When I apply it (informally) to a set of the form $M\times N \in \mathcal{B}(\mathbb{R})$, I get:
$$ P((X,Y) \in M\times N) = \int_M f_X(x) \int_N \delta(y-h(x)) \; \mathrm{d}y \mathrm{d}x = \int_{M\cap h^{-1}(N)} f_X (x) \; \mathrm{d}x $$
which is coherent with the definition \eqref{eq:1}.
Could you help me to give a formal explanation to \eqref{eq:2}? Does it define a measure on $\mathbb{R}^2$? Thank you very much!
EDIT: This document http://www.elac.pvamu.edu/Include/Math/AAM/Vol3_No1/Chakraborty%20MAA-R14-071406%20Final%20_4_%206-12-08.pdf shows a lot of applications of the Dirac distribution in probability/statistics, but it does not provide any rigorous justification to its use. Does anyone know any reference that presents it in a rigorous way?
The image measure is given by the formula $$ \mu_{X,Y}(A)=P((X,Y)\in A)=\int_{\Bbb R} 1_A(x,h(x))f_X(x)\,dx. $$ As $\mu_{X,Y}$ is concentrated on the graph of $h$, which is a Borel subset of $\Bbb R^2$ of Lebesgue measure $0$ (Fubini), $\mu_{X,Y}$ is not absolutely continuous with respect to $2$-dimensional Lebesgue measure, so it does not admit a density. The utility of the delta function formalism in this context is not apparent.