Floating point modulus precision (high multiple)

299 Views Asked by At

While looking at implementing floating point modulus in double-precision on an x86 CPU, I found the FPREM instruction and proceeded to test the practical precision in some cases, one being

1e48 mod 1.5

The FPREM implementation returns 1 as result, which looked "right" to me (as 1e48 is not a multiple of 3).

Then I checked against Wolfram Alpha but it returned 0 (http://www.wolframalpha.com/input/?i=1e48+mod+1.5%29)... now I am having doubts, is Wolfram failing or are FPREM and my expectation wrong?

Also in case Wolfram is failing, is there an online high-precision resource I could use to compare floating point modulus precision?

2

There are 2 best solutions below

0
On BEST ANSWER

The true value of $10^n$ mod $1.5$ is $1$ for all positive integers $n$. Wolfram Alpha gets it right for $n \le 16$, and wrong for (it seems) all $n \ge 17$.

Wolfram Alpha gets 1e17 mod 3 right, but 1e17 mod 3.0 wrong. So a loss of floating-point precision is to blame.

4
On

It might be worth pointing out that you can use WolframAlpha to get the correct result by simply specifying greater precision. You can specify 1.5 to 50 digits of precision like so: 1.5`50. So your query would be:

1e48 mod 1.5`50

You can even specify exact precision:

1e48 mod 3/2