Understanding floating point arithmetic with an example

Question

Im trying to understand FPA. Taking an example, suppose

$$ f(x) = \frac{ 1 }{1-x} - \frac{1}{1+x} $$

$x \neq \pm 1$. So, to compute this expression, we are using 2 divisions and one substraction, thus giving 3 operation. Are there values for which it is dificult to compute such expression in floating point arithmetic? I would guess for values near the restrictions. Is this correct?

Answer 1

A difficulty (but perhaps not the only) arises near $x = 0$. For example, analytically, $f = g$ where

$$ g(x) = \frac{2x}{1 - x^2} $$

but as $x \to 0$, $1 \pm x \approx 1$ and $1 - x^2 \approx 1$, both because the mantissa precision is limited, so $f(x) \approx 0$ but $g(x) \approx 2 x$. Clearly, analytically $f(x)\neq0$ for $x \neq 0$, so rearranging the formula symbolically before evaluation as $g(x)$ gives a more accurate result.

Note: Here I am using $\approx$ to denote floating point equality, not an approximate equality.

Answer 2

Machine $\epsilon$ is the smallest number such that $$1+\epsilon >1$$ which implies that, if $x <\epsilon$, then, at machine level, $1\pm x=1$ and, then, the problems for the kind of expression you posted.

This does not have anything to do with the smallest number of a real kind you could effectively compute.

Answer 3

Major sources of errors tend to be where you do an addition or subtraction that causes cancellation.

So, the values of $x$ to worry about are the solutions to:

• $1-x = 0$
• $1+x = 0$
• $\frac{ 1 }{1-x} - \frac{1}{1+x} = 0$

So the values to potentially worry about are when $x$ is near $1$, $-1$, $0$, $+\infty$, or $-\infty$.

Near $\pm \infty$, the two terms in the third equation have opposite sign, so although the difference is near zero, there isn't any cancellation. So, the only remaining worries are $1, -1, 0$. (and, they really are problems)