Radon-Nikodym Derivative of a Mixed Distribution

Question

When we have a continuous distribution $F_X(x)$, we can take the Radon-Nikodym derivative (RND) of the probability measure with respect to Lebesgue measure to get the density $f_X(x)$.

When we have a discrete distribution, we can take the RND with respect to the counting measure to get the mass function (so "density" in measure theoretic probability).

When we have a mixed distribution, say 50% standard normal and 50% 0s (so a jump discontinuity in the CDF at 0, see the drawing), what would be the measure to use that is some kind of hybrid of Lebesgue and counting measure?

(Starts out looking normal, jumps up at 0, then gets back to looking normal.)

Answer 1

(This is all phrased in terms of measures and not in terms of distributions, since in the comments you said you want a measure-theoretic approach.)

I think what you want to do is to break up some probability measure into parts which are easier to understand.

One of the theorems which accomplishes this is Lebesgue decomposition theorem:

Let $\mu, \nu$ be $\sigma$-finite measures defined on the same measurable space. Then there exists with respect to $\mu$ both an absolutely continuous measure $\nu_{\text{as}}$ and a singular measure $\nu_{\text{s}}$ such that $$ \nu = \nu_{\text{ac}} + \nu_{\text{s}}.$$

Here a singular measure $\lambda$ with respect to $\mu$ is a measure such that there exists a set $X$ with measure zero with respect to $\mu$ such that $\lambda(X^c)=0$. In other words, we divide the measure space into a part which we measure with $\lambda$, and another part which we measure with $\mu$.

This theorem can be refined even more to obtain $$\nu = \nu_{\text{ac}} + \nu_{\text{pp}} + \nu_{\text{s}},$$ where $\nu_{\text{ac}}$ is again absolutely continous with respect to $\mu$, $\nu_{pp}$ is a discrete measure (i.e. a sum $\sum_{j \in \mathbb{N}} z_j \delta_{x_j}$, with the $(x_j)_{j \in \mathbb{N}}$ being points in the measure space and $(z_j)_{j \in \mathbb{N}} \in \mathbb{{R_0^{+}}^{N}}$), and $\nu_{s}$ is again singular with respect to $\mu$.

Now we can use this to get a sort of "generalised Radon-Nikodým-derivative" (or a "generalised density" if you will), only that it consists out of multiple parts which are relative to several measures:

Let $\mu$ be your probability measure and $\lambda$ the Lebesgue-measure. Then we can write $$ \mu(A) = \int_A \frac{d\mu_{ac}}{d\lambda} d\lambda + \sum_{x \in \sigma(\mu_{pp})} \int_A \frac{d\mu_{pp}}{d\delta_{x}} d\delta_x + \mu_s(A), $$ for any measurable $A$.

Here $\sigma(\nu_{\text{pp}})$ denotes the support of $\nu_{\text{pp}},$ that is all points in the measure space on which the measure isn't zero (which is at most countable since it is discrete).

This essentially is what Masacroso has commented in more measure-theoretic terms.

In general you won't be able to get rid of the singular part $\nu_s$, but you could for example try to decompose that using the above theorem with respect to some other measure. But I think in the example you have in mind (a sort of superposition of a Lebesgue measure and a Dirac measure) $\nu_s$ should be zero.

EDIT: In the case of the sketch which was added to the question the measure $\mu$ looks like $$\mu(-\infty, x) = f(x),$$ where $f: \mathbb{R} \to [0,1]$ is absolutely continuous on both $(-\infty, 0)$ and $(0, \infty)$ with a jump discontinuity at $0$.

To decompose $\mu$ as above we note that for $a < b \in \mathbb{R}$ with $0 \notin [a,b]$ we have $$ \mu(a,b] = \mu(-\infty, b] - \mu(-\infty, a]. $$

To find the discrete parts we calculate the measure of the singletons $\{ x \}$ with $x \in \mathbb{R}:$ $$ \mu\{ x \} = \mu(-\infty, x] - \lim_{\varepsilon \uparrow 0} \mu(-\infty, \varepsilon] = \begin{cases} 0 & x \neq 0 \\ f(0^+) - f(0^-) & x = 0 \end{cases},$$

since we only have a discontinuity at $0$. Now for again $a < 0 < b \in \mathbb{R}$ we can write

$$ \mu(a,b] = \mu(a,0) + \mu\{0\} + \mu(0, b]. $$

Now define $\mu_{\text{ac}}(a, b] := \mu((a,b]\backslash\{0\}), $ and $\mu_{\text{pp}}(a,b] := \mu\{0\}\delta_0(a,b].$ These can be extended to full measures for arbitrary measurable sets.

Then we have $$ \mu = \mu_{\text{ac}} + \mu_{\text{pp}}, $$ with $$\frac{d\mu_{\text{ac}}}{d\lambda} = f', \ \frac{d\mu_{\text{pp}}}{d\delta_0} = \mu\{0\}.$$