Let $f$ be a function and $\mu$ be a probability measure. I've frequently seen a notation like: $\mathbb E_\mu[f]$. Does it mean that $$ \mathbb E_\mu[f]=\int_\mu f=\int f(x)d(\mu(x))? $$
I've checked a various sources but only something like (elementary) expectation operator $\mathbb E[X]=\sum_i x_ip_i$ is defined.
The first notation, $\mathbb{E}_\mu[f]$, is correct. The original definition of expectation is basically Lebesgue integral with respect to the measure $\mu$ and can be denoted as $\int f\,d\mu$, or if the integrand needs a variable $x$ to avoid confusion with others, $\int f(x)\,d\mu(x)$. Usually what is written under the integral sign is for denoting the space which the integral is taken. For example, if one takes integral over the set $K$, it is written $\int_K f\,d\mu$ or $\int_K f(x)\,d\mu(x)$.