I'd like somebody to verify my solution of the following problems:
Let's assume, that float arithmetics $fl()$ has precision $\nu$ for standard operations $(+\ -\ \cdot \ \div)$.
a.) Estimate error's upper bound for calculation $(a\cdot a + b\cdot b)\cdot c$
b.) Estimate error's upper bound for $(a\cdot a - b\cdot b)$
My solution:
a.)$$fl(a\cdot a) = a\cdot a(1+\alpha) \\ fl(b\cdot b) = b\cdot b(1+\beta) \\ fl(a\cdot a + b\cdot b) = \left (a\cdot a(1+\alpha) + b\cdot b(1+\beta) \right )(1+\gamma) $$ $$fl((a\cdot a + b\cdot b)\cdot c) = \left (a\cdot a(1+\alpha) + b\cdot b(1+\beta) \right )(1+\gamma)c(1+\delta) $$
Therefore, using arithmetics' precision $\nu$ we get:
$$fl((a\cdot a + b\cdot b)\cdot c) \le \left (a\cdot a(1+\nu) + b\cdot b(1+\nu) \right )c(1+\nu)^2
$$
$$fl((a\cdot a + b\cdot b)\cdot c) \le (a\cdot a + b\cdot b)(1+\nu)^3$$
Is it correct?
I've always had that intuition that error's exponent is proportional to the number of calculations,
whereas here we've got $4$
of them and the exponent is $3$.
b.)
Pretty much the same idea...
$$fl(a\cdot a - b\cdot b) = \left (a\cdot a(1+\alpha) - b\cdot b(1+\beta) \right )(1+\gamma)
$$
but, how should I substitute the $\nu$ here? Since $b^2$ is non-negative,
should I go like this (mind the $-$ instead of $+$)?:
$$fl(a\cdot a - b\cdot b) \le \left (a\cdot a(1+\nu) - b\cdot b(1-\nu) \right )(1+\nu)
$$
Thanks,
user2938932
For a, your intuition that you get one power of $1+\nu$ per operation is a good one. In your example, no single number gets more than three operations applied to it. The multiplies of $a \cdot a$ and $b \cdot b$ are in parallel, and each one gets only one factor. Then when you add them, you have two factors even though you have done three operations. A more extreme case would be adding $2^n$ numbers. If you add them left to right, you have $2^n-1$ operations and get that many factors of $1+\nu$. If you add them in pairs, then pairs of pairs, etc. you only apply $n$ operations to any given number, and only get $n$ powers of $1+\nu$, even though you did a total of $2^n-1$ operations.
For b, unless you have some reason to know the errors will cancel, you have to worry that they might add. In fact it is worse than that. If $a$ and $b$ are very close together, you can get cancellation in the central value of the subtraction. So the error of $(a\cdot a - b\cdot b)$ is bounded by $(a\cdot a + b\cdot b)(1+\nu)^2$, which can be much larger than $(a\cdot a - b\cdot b)(1+\nu)^2$