Why $(255.71 - 255.61 - 0.1)$ is not $0$?

Question

Why $(255.71 - 255.61 - 0.1)$ is not $0$? I used google calculator. It produces the result $-5.689893×10^{-15}$. Although, $(50.71 - 50.61 -0.1)$ produces the result $0$.

Answer 1

This is due to floating point error. Informally, computers and calculators store numbers in binary only up to a certain precision; just like how $\frac{1}{3}$ does not terminate in base 10, some finite decimals do not terminate in binary. Small rounding errors from the loss of precision caused by using only a relatively small number of bits can compound and offset calculations, giving results like the one you noticed. Also note that the answer it gives you is insanely close to 0 (written explicitly, this would be $-0.000000000000005689893$).

Answer 2

Calculators have weird rounding errors owing to their use of binary instead of base 10. It's a complicated issue, and not trivial, and I don't feel confident in my abilities to go into it at length, but I will touch on one of the core issues causing this.

Part of the issue is that what may be a terminating decimal in base 10, may not be one in base 2 or other bases. If you can write a number in base $b$ in terms of the prime factors of $b$, it will terminate as a decimal. In base 10, that means any number of the form $2^a 5^b$, for integers $a,b$, will terminate; if the number has any other prime factor, its decimal doesn't terminate. However, in base 2, only numbers of the form $2^a$, for integer $a$, terminates.

(The number zero is an obvious and trivial exception to the above rules.)

How is this a problem? Well, depending on the calculator, a terminating number may be stored just as a truncated decimal, and in binary. For example, $1/3$ won't be stored as $1/3$ in a relatively basic calculator; it'll be stored as the binary equivalent of $0.333333333333$, cutting off after ever-so-many decimal places. With certain sequences of operations, this can cause rounding errors, resulting in nonzero answers when they shouldn't appear.

This compounds with the "terminating decimal" issue above since very few numbers in binary terminate - most are repeating decimals (maybe periodic, maybe not). And since calculators work in binary ... see the problem?

Granted this is a very broad overview of (my understanding of) the problem, but I think I'm touching on the right idea. Of course, certain algorithms and computation methods also have ways to counteract this - it's, in part, an issue of how your calculator/algorithm/programming environment/whatever handles these.

Answer 3

Even some apparently easy calculations can have some rounding error when performed in a machine. This is because the numbers are not always stored exactly in the memory. This comes from two facts at least:

• the number of digits stored for each numeric value is finite, so the machine will discard digits exceeding that number and will truncate or round for the last digit;
• these values are stored in binary system, so it can happen that a number that seems simple and with a finite writing in base ten is periodical in base two and has to be approximated. (*)

The combination of these factors will lead sometimes to approximate but not exact results, and sometimes to exact results. In the second case the result is exact, and in the first is a very good approximation to zero (-5.689893e-15 is actually $$-0.000000000000005689893,$$ which you can take in this case as if it were zero.)

---

(*) Just to give an example, a number so inocent as $0.2$ is periodical in base 2: $$0.2=0.\overline{0011}_{|2}=0.00110011001100110011..._{|2},$$ this means that eventually it will be cut and the stored number will be close to but not exactly $0.2$. Just to give some examples, $$0.0011_{|2}=0.1875$$ $$0.00110011_{|2}=0.19921875$$ $$0.001100110011_{|2}=0.199951171875,$$ and so on.