Let $f(x)=x.$ If the units of $x$ are meters, then the area under the curve $y=f(x)$ between $x=1$m and $x=2$m is $\frac12(2\text m)^2-\frac12(1\text m)^2 = 1.5\text m^2.$
Now let $f(x)=x^2.$ Again, the units of $x$ are meters. This time, the area under the curve $y=f(x)$ between $x=1$m and $x=2$m is $ \frac{(2\text m)^3}{3} - \frac{(1\text m)^3}{3} =3.5\text m^3.$ The area gets expressed in cubic meters. Where is the mistake?
What happens with the units when the function is $f(x) =e^x$ or $f(x)=\frac{1}{x}$?
When you think of the integral $$ \int_a^b f(x)dx $$ as the area under the graph of $f$ over the interval $[a,b]$, the variable $x$ has units Length. So does the dependent variable $y$, whose value at $x$ is $f(x)$: the height of the graph at that point. That is independent of the unit calculations suggested by a formula for $f$.
Since $dx$ (which represents a change in $x$) also has units Length, the integrand $f(x)dx$ has the units $\text{Length}^2$ as expected.
In the example $$ \int_a^b x^2dx = \frac{b^3}{3} - \frac{a^3}{3} $$ you can think of $$ \frac{b^3}{3} $$ as $$ \frac{b^2}{3} \times b $$ which will have units $\text{Length}^2$ since the numerator of the fraction has units Length.
In an application where $f(x)$ is the velocity at time $x$ the integrand has units $$ \frac{\text{Length}}{\text{Time}}\times \text{Time} = \text{Length} $$ as expected for the distance covered during the time interval.