What is epsilon algebra and why is it important in Numerical Analysis?

Question

My professor is using the following slides:

What is epsilon algebra and why is it important in Numerical Analysis?

Answer 1

This is a question you should ask your professor. If they have not made it clear, then many students in the class are probably wondering about that.

The different epsilons represent small errors. That "epsilon algebra" is nothing but first order approximation of errors. All terms including products of epsilons is considered small and is thrown away. All that is left is the main term without any epsilons and terms linear in the epsilons.

When errors are small, such calculations give a decent approximation of propagation of errors. Errors are inevitable in numerical analysis, and it is important to understand how big an error you make in a calculation if you start with a given error. There are a number of sources of error, so it is convenient to have several epsilons.

Answer 2

The is a variation of dual numbers and derivations is its generalization to several variables. They are important because the algebra tracks the propagation of errors in arithmetic and other numerical operations giving a good approximation to how much error is contained in the results of the operations. It is a way of automatically keeping track of derivatives which determines how sensitive the operations are in outputs relative to inputs. You can think of it as the first order approximation to a function of several variables.

The specific variation used is close to significance arithmetic in that we replace every variable $\, x_i \,$ with $\, x_i(1+\epsilon_i) \,$ where each $\, \epsilon_i^2 = 0, \,$ also $\, \epsilon_i \epsilon_j = 0, \,$ and assuming that each $\, x_i \ne 0. \,$ It is easy to verify the operation $\, x_i(1+\epsilon_i)\, x_j(1+\epsilon_j) = x_ix_j(1 + \epsilon_i + \epsilon_j) \,$ Notice how this is different from using $\, x_i + \epsilon_i \,$ where we allow $\, x_i = 0 \,$ and where $\, (x_i + \epsilon_i)(x_j + \epsilon_j) = x_i x_j + (x_i \epsilon_j + x_j \epsilon_i). \,$ Also notice that if $\, x_i = x_j = x, \,$ then $\, x_i(1 + \epsilon_i) - x_j(1 + \epsilon_j) = 0 + x(\epsilon_i - \epsilon_j)\,$ but the difference has lost all significance.

The epsilon algebra is a simple method compared to the more advanced interval arithmetic which can be used to put bounds on rounding and measurement errors in computation. The advantage of epsilon algebra is that simple rules of calculation are sufficient compared to the complicated calculation rules of interval arithmetic.

Answer 3

The first thought which comes to mind are those of error or estimation of approximation. How far is an iterative process of the supposed limit object.

If you have a small quantity, say $\varepsilon=10^{-4}$, then $\varepsilon^2=10^{-8}$ is even smaller. Worst for $\varepsilon^n$ if $n\ge 3$.

Answer 4

To take one example from the basic rules, note that

\begin{align} (1+\varepsilon_1)(1+\varepsilon_2) &= 1+\varepsilon_1+\varepsilon_2+\varepsilon_1\varepsilon_2\\ &\cong 1+\varepsilon_1+\varepsilon_2 \end{align}

This is a reasonable assumption when epsilon is so small that products (or powers) of epsilons can be ignored for the purposes of the task.