I've got a random variable $X\sim P$ which can be discrete or continuous.
From a measure theoretic point of view:
- Is it correct to talk about proability density function even in the discrete case?
- How do I write the expectation of a generic function $h(x)$ over the probability measure? Is this notation $$ \mathbb E[h(X)]=\int h(x)\,\mathrm{d}P(x) $$ correct in both the discrete and continuous cases?
In case of discrete random variables one usually talks about probability mass function (pmf) instead, which is given by $$ p_X(x) = \Bbb P(X = x) $$
In case of continuous random variables one talks about probability density functions (pdf) $f_X$ characterized by the following condition that should hold for every measurable set $A$ (or at least any open interval) $$ \int_A f_X(x)\mathrm dx = \Bbb P(X \in A). $$
To be formal in measure-theoretic sense, one should specify a density w.r.t. which measure are we talking about. For example, in case of continuous random variables, its pdf is exactly the density of its distribution measure w.r.t. the Lebesgue measure (the length of intervals) $\mathrm dx$. Since the Lebesgue measure is somewhat canonical on the real line, people omit saying that pdf is the density w.r.t. Lebesgue measure and just say density.
Interestinly enough, the pmf is also a (measure-theoretic) density, but not w.r.t. the Lebesgue measure. Instead, it is a density of the distribution measure of a discrete random variable w.r.t. counting measure (that assigns the number of elements to finite sets and is infinite over infinite sets).
So, formally distribution measures $P_X$ of both discrete and continous have densities w.r.t. relevant base measures (counting or Lebesgue), but it is common to refer to those densities as probability mass functions and probability density functions respectively. You can stick to that as well until you start exploring formal measure-theoretical probability.
Finally, formula $$ \Bbb E[h(X)] = \int_{\Bbb R} h(x)\,P_X(\mathrm dx) $$ holds true for any random variable $X$: discrete, continuous, not discrete but also not continuous since the integral here is the Lebesgue integral which it taken w.r.t. the distribution measure $P_X$. In particular, for continuous random variable you get $$ \int_{\Bbb R} h(x)\,P_X(\mathrm dx) = \int_{-\infty}^\infty h(x)\,f_X(x)\mathrm dx $$ and for discrete random variable you get $$ \int_{\Bbb R} h(x)\,P_X(\mathrm dx) = \int_{\Bbb R} h(x)\,p_X(x)\#(\mathrm dx) = \sum_{x: P_X(x) > 0}h(x)p_X(x) $$