If I have $f(a,b)=a+b$ and first I have to find the condition number:
I got to $(cond f)(a,b)=\dfrac{a}{a+b}+ \dfrac{b}{a+b}=1$ But the rest of the question says to show the subtraction of close numbers will cause instability in this question? How can I do it?
Let $\text{fl}(x+y)$ the floating point representation of the sum of $x$ and $y$, where actually it holds $$\text{fl}(x+y)=\text{fl}(\text{fl}(x) + \text{fl}(y))$$ Compute the relative error:
$$\epsilon = \frac{|x+y - \text{fl}(x+y)|}{|x+y|} \leq \frac{|x|}{|x+y|}\epsilon_x + \frac{|y|}{|x+y|}\epsilon_y $$
where $\epsilon_x =\frac{|x - \text{fl}(x)|}{|x|}$ is the relative error in $x$.
The computation above shows that, whenever $ x +y \approx 0 $ you have large relative error as the denominator becomes larger and larger. This is exactly what your question is asking.