I am really confused over the idea.
If I have a circle I can approximate its area by using a regular polygon inside of it, with $n$ sides, and I can just split that polygon into triangles and compute the area. If I want better accuracy then I can use a polygon with more sides. The accuracy becomes better since the area between the polygon and circle (the "error") decreases as $n$ increases.
So it stands to reason that if we could keep adding sides our answer would keep getting better since the error would get closer to $0$.
I don't know what it means to "add infinitely many sides" because to me the idea doesn't make sense. No matter how many sides there are we can always add one more. But it does make sense to ask what's the value we never actually reach but get closer and closer to? For that we use the concept of a limit.
The error becomes "infinitely small" but I don't understand what this really means. Is "infinitely small" the same as $0$? Because by definition it's never quite getting to $0$. But conceptually then how can we say something with nonzero error gives us an exact answer as if it had $0$ error?
Infinitely small error and $0$ error surely aren't the same thing.
Even though it seems like you already grasp the concept, let me reword part of the train of thought exposed:
Draw a circle of radius $1$ and fix a number $\epsilon > 0$. Make it really as small or as big as you want, and now ask me to create a regular polygon whose area misses the exact area of the circle by less than $\epsilon$. If you make $\epsilon$ really big, then I won't need to draw a polygon with many sides but if you make $\epsilon$ really, really small, then I may need to draw a polygon with a very big number of sides! Either way, we can agree that whatever $\epsilon > 0$ you choose, my task isn't impossible. Right?
Thus you probably can't say that my best approximation to the area of a circle has error $\delta$ with $\delta > 0$. Why not? Just set $\epsilon = \delta$ and build a polygon with approximation error inferior to $\epsilon$. So if you really insisted on finding a number that you could say "the approximations by polygons have error blah", then no positive number would do... so it is just like $0$ is the first number that isn't just wrong... or it is like $0$ is the less wrong answer, in a sense...
That is the way I would explain this relation, but certainly infinitely small and $0$ aren't the same thing.
In a not-so-precise sentence, infinitely small is not $0$, but it is as if it were for all practical purposes.