Imagine dividing a sphere into concentric spherical shells of thickness $dr$ and inner radius $r$.
The volume of each shell is $$dV = \frac{4\pi}{3} [ (r + dr)^3 - r^3]$$
Expand the cubic expressions, we get:
$$ (r + dr)^3 - r^3 = r^3 + dr^3 + 3r^2 dr + 3rdr^2 - r^3 = 3r^2dr + 3rdr^2 + dr^3 $$
Assuming that $dr^3 = 0$ and $rdr^2 = 0$, we get:
$$(r + dr)^3 -r^3= 3r^2dr$$ Thus, the volume of each shell is $dV = 4\pi r^2dr$.
If we integrate along the radius, then we get $$\int_0^R 4\pi r^2dr = \frac43\pi R^3$$ This confirms that our analysis of the spherical shell volumes is correct. However, this analysis relies on the assumption that $dr^3 = 0$ and $rdr^2 = 0$. My question is why are these assumptions correct? If we assume those values are zero, shouldn't the final value just be approximately correct by an infinitesimal amount instead of being absolutely correct?
Instead of thinking of $dr$ as a number, think of the manipulations you've used to get $dV=4\pi r^2dr$ as a shortcut for computing $\frac{dV}{dr}$ using the limit definition of the derivative: $\lim_{h\to0}\frac{3r^2h+3rh^2+h^3}h=3r^2$. What you've calculated isn't really the volume of any particular shell, but the limit of the ratio $\frac{\text{shell volume}}{\text{shell thickness}}$, as the thickness approaches $0$.
Then you're using the fact that $\int_0^R\frac{dV}{dr}{dr}=V(R)-V(0)$. You've basically just taken a derivative followed by an antiderivative.
The formula above is exact because of the fundamental theorem of calculus. The integral is (by definition) the limit of a sequence of approximations obtained by subdividing the domain into smaller and smaller intervals (corresponding to thinner and thinner shells in your example), and it turns out that if you estimate each shell volume in this process by $\Delta V=\frac{dV}{dr}\Delta r$, then the sequence of approximate volumes approaches the exact total volume you want in the limit, because the "error" corresponding to the "erased" higher-degree terms approaches $0$.