I am leaning numerical analysis and getting a hard time to understand cancellation error. For example, suppose we have $\ln(x+1)-\ln(x)$, this is the same as $\ln\left(\frac{x+1}{x}\right)$. Suppose we just need 6 digits when we compute $\ln(x+1)-\ln(x)$. I have tried to MatLab to look at 0.01,0.5,100,1000, etc, I don't see that how to compare $\ln(x+1)-\ln(x)$ and $\ln\left(\frac{x+1}{x}\right)$ because the results from MatLab are the same.
Can someone tell me how to identify when cancellation error exists? Thanks.
The problem will arise when $x$ is huge. In this case $\ln(x+1)-\ln(x)$ is a subtraction of nearly equal numbers which will give instability. In particular, when $x$ is huge, $\ln(x+1)-\ln(x)$ is very nearly $1/x$; the true error in that approximation is less than $1/x^2$. With the second version, you do get very nearly $1/x$, with the first you do not. Try it out with $x=10^{14}$.
Unfortunately this example is not perfect because when $x$ is large, the second implementation suffers from another source of floating point error, namely "swamping". This can be very clearly seen when $x=10^{16}$, where both methods (in double precision) give zero. In the second implementation this happens because 10^16+1==10^16, in the sense of floating point arithmetic. A better implementation for large $x$ would be log1p(1/x).