Consider the following definition given by the author of my book:
The definite integral of the function $f(x)$ on the interval $[a,b]$ is the limit of the integral sums $$\int_a^b f(x) dx = \lim \sum_{i=0}^{n-1} f(\xi_i) \Delta x_i \text{ when }\max |\Delta x_i|\to0.$$
Where $\xi_i\in[x_{i-1},x_i]$ and $\triangle x_i=x_i-x_{i-1}$. In order for the Reimann sum to be equal to the actual area under the curve the interval $\triangle x_i$ must become infinitesimaly small. But I don't understand how the notation used by the author conveys this idea. Please explain.

For each $i=1,...,n-1$, the notation $f(\xi_i)\Delta x_i$ can be interpreted as the area of a rectangle of base $\Delta x_i$ and height $f(\xi_i)$.
Thus, the notation $\displaystyle \sum_{i=0}^{n-1}f(\xi_i)\Delta x_i$ (sum of the areas of the $n-1$ rectangles) can be interpreted as an approximation to the actual area.
As you said, in order this sum to be equal to the actual area, the interval $\Delta x_i$ must become infinitesimally small (for all $i=1,...,n-1)$. This part is covered by the condition $\max|\Delta x_i|\to 0$.
Explicitly, the definition $$\int_a^bf(x)\;dx=\lim\sum_{i=0}^{n-1}f(\xi_i)\Delta x_i\text{ when }\max\Delta x_i\to 0$$ means
According to the previous interpretation, this means that if the base of all rectangles are small enough, then the difference between the actual area and the approximation is very small.