Wikipedia states the following (can be found at the bottom of this section):
However, differential entropy does not have other desirable properties:
It is not invariant under change of variables, and is therefore most useful with dimensionless variables.
Where differential entropy is as defined here:
Let $X$ be a random variable with a probability density function $f$ whose support is a set $\mathbb{X}$. The differential entropy $h(X)$ is:
$$h(X) = -\int_{\mathbb{X}} f(x) \log f(x) dx$$
I would like to understand why it is not invariant under change of variables, but doesn't such a claim depend on $f$ and on the function used for the change of variables? I'm only familiar with the classic theorems (e.g. as in Wikipedia here) which don't seem to produce such a general claim... Or is it that there's a simple way to demonstrates it wouldn't hold for a large class of functions?
I think the statement is meant as "in general not" statement, but however: One of the simplest transforms one could think of is to look at $Y:=2X$, which has pdf $f_Y(y) = \frac12 f_X(y/2)$. But the differential entropy of $Y$ is \begin{align} \int_{2\mathbb X} f_Y(y)\log f_Y(y)\mathrm dy &= \int_{2\mathbb X} \frac12 f_X\left(\frac y2\right) \log\left(\frac12f_X\left(\frac y2\right)\right)\mathrm dy \\&= \int_{\mathbb X}f_X(x)\left(\log\frac12+\log f_X(x)\right)\mathrm dx \\&= \log \frac12 + \int_{\mathbb X}f_X(x)\log f_X(x)\mathrm dx. \end{align} As you see, just scaling already gives us an extra term of $\log\frac12$. As you see, the problem is the $\frac12$ appearing in the logarithm. This is remedied by the Limiting density of discrete points, where there is an additional probability density $m$ on $\mathbb X$ in the logarithm: $$ \int_{\mathbb X} f(x)\log\frac{f(x)}{m(x)} \mathrm dx. $$