Finding Relative Error Of $\cos^2x-\sin^2x$

Question

Find the relative error when storing six digits in the mantissa using rounding of $\cos^2x-\sin^2x$ where $x=0.7854$

So $f(0.7854)=-3.673205105\cdot 10^{-6}$

And $fl(0.7854)=(0.7071)^2-(0.7071)^2=0$

So the relative error is $$\frac{\mid -3.673205105\cdot 10^{-6}-0\mid}{\mid -3.673205105\cdot 10^{-6}\mid}=100\%$$

But the answer in the book is $36%$ where did I get it wrong?

Answer 1

You need to round all intermediate values to 6 mantissa digits, for some reason you only used 4:

For the exact result, as you said

cos(2*0.7854)
 = -3.6732051033465739e-06

For the first step, the trigonometric values are

(cos(0.7854), sin(0.7854))
 = (0.70710548251123628, 0.70710807985947355)

Now round to 6 digits after the decimal dot and square

(0.707105**2, 0.707108**2)
 = (0.49999748102499997, 0.500001723664)

and compute the difference of the squares, rounded again to 6 digits

0.499997-0.500002
 = -4.999999999921734e-06

The relative error is thus

-4.999999999921734e-06/-3.6732051033465739e-06-1
 = 0.36120904203425686

that is, the announced $36\%$.