Informally, I understand pdfs as the "shape" of relative probabilities in continuum of range space of an RV scaled down to have an integral over $(- \infty, \infty )$ equal to one
Can we then theoretically prove that all possible relative probability densities can be represented by such a function / "shape"?
One way I've been able to formalize this question is as follows:
Let $X$ be a random variable.
Given a function $r(x_1, x_2)$ which represents relative probability of the event $X = x_1$ relative to that of event $X = x_2$ where $x_1, x_2 \in range(X)$, can we come up with a PDF $f$ such that $f(x_1)/f(x_2) = r(x_1, x_2)$?
$r$ will have the following properties which might be useful in proving this:
- $r(x_1, x_2)$ is defined only when the event $[X = x_2]$ is not an impossible event
- $r(x_1, x_2) = \frac{1}{r(x_2, x_1)}$ provided $r(x_2, x_1) \neq 0$
- $r(x, y)r(y, z) = r(x, z)$
Assume that the the range of the random variable does not have discrete points. It may be a union of ranges $(a, b)$, $(a, b]$, $[a, b)$, $[a, b]$ where $a \neq b$
In the usual sense: no. A discrete random variable for instance has no probability density in the usual sense.
In a broader sense: yes. Given any CDF, the distributional derivative of that CDF (in the sense of distribution theory, not probability theory) is a PDF, in a loose sense. For example, the "probability density" of a discrete random variable is a convex combination of Dirac delta functions at different points.
Incidentally, your $r$ formalism has lots of subtle issues, in particular "impossible event" is much more badly defined than you would expect. Moreover PDFs are extremely non-unique, so you will need to specify some "selection criterion" to pick one. A starting point for such a criterion is continuity at all Lebesgue points of the CDF, but there are still problems at boundaries (for example there is not really a well-defined value for the PDF of an exponential random variable at $x=0$).