The density of states of a system in an interval $[E, E+dE]$ is given implicity by $dV = D(E)dE$ (Or I suppose explicitly, by $D(E) = \frac {dV}{dE}$, but we'll be integrating it anyway, so it doesn't really matter).
Is there some way of stating this without using the infinitesimal in the interval? I hate when physicists throw infinitesimals around and usually can find a more mathematical statement, but what could I do here?
On
When a working physicist writes $dV=D(E)\>dE$ then this is meant as a heuristic manoeuvre, and he is not expecting from you that you are going to interpret this in terms of clean differential geometry, let alone of nonstandard analysis. The intended interpretation is the following, and reading these lines you can surmise how much content is packed into the simple formula at the begin:
There is a certain $E$-interval $\Omega\subset {\Bbb R}$, and for each reasonable subset $I\subset \Omega$ a "volume" $V(I)$ of states is defined. This means that $V(\cdot)$ is a measure on $\Omega$. In the case at hand we can encode this measure into a density $D:\>\Omega\to{\mathbb R}$. This density is related to $V(\cdot)$ via the formula $$V\bigl([a,b]\bigr)=\int_a^b D(E)\>dE\qquad \forall\ [a,b]\subset\Omega\ .$$ It follows that the function $\phi:\>x\mapsto V\bigl([a,x]\bigr)$ satisfies $$\lim_{h\to 0+}{V\bigl([a,x+h]\bigr)-V\bigl([a,x]\bigr)\over h}=D(x)\ ,$$ which then can be condensed into ${dV\over dx}(x)=D(x)$.
On
The physicist Pierre Duhem wrote:
Consider a continuous sequence of states of the same system, we fix the attention on these different states in the order that allows us to switch between them continuously. To identify this intellectual operation to which we submit all the mathematical schemes used to represent the set of concrete bodies, we say we impose on the system a virtual change.
[...]
Changes in the numerical values of the variables used to define the state of the system must be compatible with the conditions that logically result from the definition of the system, but only with these conditions. And in particular, the changes in numerical values may well contradict the experimental laws that govern the system of all the concrete bodies that our abstract mathematical system has the duty to represent.
(quoted in Capecchi's History of Virtual Work Laws ch. 18.1 "Pierre Duhem’s concept of oeuvre", p. 397; originally from Duhem's 1911 Traité d’énergétique ou de thermodynamique générale)
These infinitesimals (and many others that appears in physics) can be interpreted as finite differences, $\Delta E$ for instance. Thereby, the density obtained using $\Delta V=D(E)\Delta E$ would be an average density. To obtain the density you want, just take the limit $\Delta E\to 0$ of this equation.
Note that, infinitesimals and finite differences are not the same thing. But on a scientific point of view the error you get when using one instead of the other is acceptable since every experimental measure has an intrinsic error associated to it.
On the other hand, infinitesimals don't have a rigorous definition in standard analysis, although some people claim that they can be put in a rigorous setting by using non-standard analysis, but I don't know nothing about it. If you forget for a moment the concept of infinitesimal you learnt, differentials (not infinitesimals) are just linear functionals, or more generally, 1-forms. Therefore, I suggest you either go further and study non-standard analysis or just forget about the term "infinitely small".