I don't know stiff equation definition

Question

My text book is proving very difficult.

The definition of a 'stiff system' in the text book is vague.

Could someone explain the definition of a 'stiff system'?

Also: why $y'=\lambda y, y(0)=\alpha,$ where $\lambda < 0$ is a test equation?

Moreover what is the relation between a stiff system and a test equation?

Answer 1

"Stiff" is more a philosophical idea than a hard mathematical concept. We can say what "nice" equations are, for instance globally Lipschitz ODE with a low Lipschitz constant. These then obviously are "non-stiff". ODE with exact solutions that diverge to infinity in finite time fall under "certainly stiff". Many ODE with polynomial (degree $\ge2$) right sides show this behavior.

Characterization on numerical fixed-step methods

Another point-of-view is that "stiff" is any ODE that is not "nice" to numerical methods, which coincidentally is also the main application area of this notion. To define what "nice" here means, a "nice" ODE attacked with a fixed-step solver has essentially 3 modes for the global error depending on the step size.

• For (very) large step sizes the error is non-linear and can be "chaotic" due to non-linearities of the equation.
• Then for medium small step sizes the error behaves approximately as the order dictates, that is, it is of size $C·h^p$ with an almost constant $C$.
  • In a loglog plot of error vs. step size, this gives a line of slope $p$. (example)
  • In a plot of the graph of the error vs. time, divided by the expected size $h^p$ you get a function that is visibly almost the same for all middle step sizes. (example, last plot)
• For (very) small step sizes, the accumulation of floating point errors over the $O(1/h)$ many single steps dominates, the error is random with a mean of size $O(\mu/h)$ where $\mu$ is the machine constant of relative precision of the numerical data type.

Now "nice" behavior is when the middle part stretches over 3 or more "octaves", that is, there are step sizes $h$ where $h, h/2, h/4$ and perhaps also $h/8$ fall into the middle range.

An IVP can be considered "stiff" for the method and number type used if it (essentially) does not have that middle "working" interval of step sizes, that is, it moves directly from the randomness due to the non-linear nature of the problem to the randomness of floating point error accumulation, with less than 2 octaves of nearly linear behavior with slope $p$ in the loglog plot. Which means it is hard to find this middle range with minimal effort (computing a sensible loglog plot is a huge effort in this context).

About the test equation

The test equation $y'=λy$ is a nice non-stiff equation. The exact solution for $Re(λ)<0$ falls to zero, a numerical solution should have the same behavior. This serves mainly to distinguish "much too large" and "moderately large" step sizes and to define stable methods where no step size is too large. Usually this stability boundary is far above the middle range of step sizes.

For the example of an explicit Runge-Kutta method of order $p$ this stability condition can be formalized as $|1+z+\frac12z^2+...+\frac1{p!}z^p|<1$ for $z=λh$, while the boundary between large and middle step sizes can be found where the next term, the leading error term, starts to dominate the other following terms, $\frac1{(p+1)!}|z|^{p+1}>2\frac1{(p+2)!}|z|^{p+2}\iff|z|<\frac{p+2}2$.

Remark on adaptive-step-size methods

In the situation where the "fast" components of the system are close to their equilibrium position within the given error tolerances, one would expect that the step size is mostly controlled by the "slow" moving components.

However, if the method used has a bounded region of stability (chiefly all explicit methods), and the product of step size and eigenvalue for the fast component leaves this region, the method step becomes expanding for this component, it will move away from the equilibrium. At some point this influences the error estimate and the step will be reduced again. This strongly binds the maximal step size to the largest eigenvalue (with negative real part). This is largely independent of the given error tolerances, reducing these only moves the oscillation of the fast component closer to the equilibrium, but does not change the step size.

Implicit solvers have a larger stability region or no restrictions at all on stability, so that after the convergence of the fast components the step size is controlled by the slower components and the relation of maximal step size, number of steps and error tolerances will follow a more intuitive pattern.

Again, an ODE system where there is a marked difference as described above is called "stiff", if there is a visible effect depends on the method used.

In literature

In the Hairer-Nørsett-Wanner books for instance an ODE is sorted as "stiff", judging by the assignment of topics to the two volumes, if it is stiff in the above sense for the standard explicit methods