In the study of number systems we learn the axioms of real numbers.
For example:
The commutative axiom $X.Y = Y.X$
The distributive axiom $X.Z + Y.Z = (X+Y).Z$
Well, the thing is that we are working with mathematical entities, we can add $X$ with $Y$ and form a number $Z$, we can add $X$ $y$ times and call it $X.Y$, which equals to $Y.X$.
Why do we assume these properties extend to objects that aren't mathematical entities?
For example: $2.apples + 4.apples = (2+4).apples = 6.apples$, we are using the distributive axiom here.
We can't add a number $X$ to $apple$ and call it a number $Z$, or add $X$ apple times and make a number $Z$, that proves they don't work like normal numbers, but we assume it behaves like mathematical entities to make equations and calculations. Why we can safely assume it works out all the time?
Famous discussion of basically this: The Unreasonable Effectiveness of Mathematics in the Natural Sciences.
In short, it's an empirical fact that abstract numbers seem to model the world well. Collections of apples behave like sets of numbers do, in certain respects - in particular, they seem to have a corresponding notion of "cardinality" which is (or is very like) a natural number.
Note that there are all kinds of mathematical structures which do not model the world well - for example, the cardinal $\aleph_{\omega_{\omega}}$, which is too big to have much to do with the world at all. But the blind forces of evolution have discovered the abstraction of "number", which resides in the world of mathematics. That particular abstraction survives in us because of two apparent facts: the universe apparently supports scientific induction (so abstraction is helpful), and there is a particular abstraction which captures the notion of size-of-a-set.
There's no universally-convincing logical argument for why numbers should apply to the real world; it's just a universally-agreed-upon observed fact. One could imagine a world in which the mathematical abstraction of "number" is not a useful notion; any world in which scientific induction is unhelpful is such a world.