Calculator Confusion using Windows and Androids

125 Views Asked by At

I just came across this calculation, it's really confusing. I ran the following calculations in windows and android calculators.

$1/98\cdot 98 = 1$

$1/98 = 0.0102040816326531$

$0.0102040816326531 * 98 = 1.000000000000004‬$

I don't understand, I did the same calculation, but with the returned values.

1

There are 1 best solutions below

0
On BEST ANSWER

It's one of a number of common examples of roundoff errors in computers. The issue is that, in computers, the math is essentially handled in binary with finite expressions, and in binary we cannot represent fractions exactly unless it's a power of $2$.

(It's similar to how we are only guaranteed a finite, nonrepeating decimal expansion in base $10$ for a rational number $p/q$, $q \ne 0$, if $q = 2^a 5^b$ for some integers $a,b$.)

Thus in computers, unless we use some sort of special techniques, the decimal part of the integer is going to be truncated at some point if the numbers we use aren't perfect powers of $2$ - that means the number we use is not perfectly accurate. These inaccuracies can crop up over repeated calculation - how quickly they pop up in your results and the nature of said errors depends on how the math is handled in the underlying programming.

Granted, this is a high-level overview of the underlying mechanics. I don't think I'm qualified to discuss the details, computer science is far from my forte.

There is a Computerphile video that also discusses this issue if you're interested.