How is the structure of a dynamical system identified empirically?

Question

There are many examples in science in which a dynamical system, usually presented as a system of differential equations, is presented and claimed to be derived "empirically". I believe the heat equation is one well-known example, but the one I'm really interested in is the Hodgkin-Huxley neuron model, which is made up of four coupled, nonlinear ODEs that are not at all obvious.

Is there a field of study or something that has general techniques for fitting dynamical system equations to data? Even just techniques for exploring some of the structures, like specific types of nonlinearity or discontinuities would be handy.

Answer 1

I can think of splitting the approaches into two: bottom-up and top-down (but there are approaches I'm not aware of, like those in Evgeny's comments).

1. By bottom-up approaches I mean writing down equations by deducing them from first physical principles. This is the area of mathematical modelling. I concur that it is often said it's more of an art than a science. In other words, it's not exactly an algorithmic activity. You need to know something about the relevant natural sciences and need to be able to decide which effects are important enough to include in a model and which can be ignored (e.g. air resistance for low-velocity motion). Examples:

   • Harmonic oscillator. Newton's second law says that acceleration is proportional to force. The force exerted by a spring is proportional to $-1$ times the displacement from the resting position. This allows you to write down an ordinary differential equation that connects acceleration and position.

   • Reaction rate equation. The law of mass action postulates that in a chemically reacting system, the rate at which a reaction occurs is proportional to the concentrations of the reacting chemical species in the solution. You can use this to write down an ordinary differential equation for the rate of change of species' concentrations as a function of the species' concentrations.

   • Hodgkin–Huxley model. It's not simple but there is an explanation what each term does in the equation.

   In this approach you write down a set of equations and fit it to data, that is, determine the values of parameters of the model to get a close correspondence to the data. There are many issues that are studied: model discrimination and invalidation of incorrect models, and optimal experiment design to achieve these goals.

   A classic on the subject of modelling in biology is James D. Murray, Mathematical Biology, I. An Introduction. Springer-Verlag Berlin Heidelberg, 2002 (alternative link).

   A good review of identification in systems biology is Alejandro F. Villaverde, Julio R. Banga. Reverse engineering and identification in systems biology: strategies, perspectives and challenges. J. R. Soc. Interface, 2014, 11, 20130505.

2. By top-down approaches I mean what is called system identification in control engineering:

   `In control systems applications, the objective of engineers is to obtain a good performance of the closed-loop system, which is the one comprising the physical system, the feedback loop and the controller. This performance is typically achieved by designing the control law relying on a model of the system, which needs to be identified starting from experimental data. If the model identification procedure is aimed at control purposes, what really matters is not to obtain the best possible model that fits the data, as in the classical system identification approach, but to obtain a model satisfying enough for the closed-loop performance.'

   A classical textbook is Lennart Ljung, System Identification: Theory for the User. Prentice Hall, 1997, but it's fairly old.