In general, my question is: what do we mean when we say that a mathematical statement is true only in the limit as $x$ goes to $0$?
For example, if the velocity $v$ of a particle is a function of time $t$, then the particle has different velocities at different instants of time. But, we say that $\Delta s=v\Delta t$, where is $\Delta s$ is the distance traveled, is true only if the velocity is not changing during that time interval, and this condition is only true in the limit as $\Delta t$ goes to zero.
How can the condition that the velocity is not changing in the interval $\Delta t$ be true when the velocity is different at different instants of time? Since I am not able to understand how the velocity can be constant (maybe it is approximately constant) for an interval $\Delta t$ as it goes to zero, therefore I think I don't really understand generally what we mean when we say that something is true only in the limit as something else goes to zero.
This phrasing is not completely rigorous. In general when someone says that $f(x)=g(x)$ in the limit $x\to 0$, they mean something like $$ \lim_{x\to 0}\frac{f(x)}{g(x)}=1 \qquad\text{or}\qquad \lim_{x\to 0}(f(x)-g(x))=0 $$ but these conditions are different, and it is possible for two functions $f$ and $g$ to satisfy one of them but not the other (or vice versa).
In general the meaning is that $f(x)$ and $g(x)$ may not be exactly equal, but they can be made to be as "close to each other" (in some appropriate sense) as you want to by selecting $x$ sufficiently small.
Judging from your choice of example, I think what you need might be a good introductory calculus textbook ...