Online optimization

Question

What is the difference between online and offline optimal control? I got confused in concept. I am working on an stochastic optimization algorithm that is based on dividing the time horizon into equal time intervals and solving a finite horizion optimization for each intervals,then I want to expand this algorithm to optimal control, when can I claim that my method is online?

Answer 1

The term comes from the machine-learning community, and is simply a variation of the name online learning. The major difference between this and offline learning is that you feed your data-sets in sequentially instead in a single batch process. For optimisation, it often means that you are doing optimisation with a model you know to be inaccurate, and each step you want to both perform your optimisation as well as improve your model (essentially you do online-learning of your model), for example by checking the predicted state with the actual state, or the predicted cost with the actual cost.

Note also that the terms online and offline optimisation are sometimes used less formally to simply mean run-time or not. I.e. a common challenge in model-predictive control is that computing the solutions can sometimes be too slow: then a possible approach is to compute solutions for the entire state-space of interest offline, and then simply implementing it as a look-up table. The challenge here is computing the entire state-space will be infeasible for many systems of interest (especially if the state-space is continuous) due to the curse of dimensionality.

Answer 2

I believe it is fair to use the term online for your method since using the finite horizon window, you only calculate the optimal solution within your time horizon. As such, your controller may (and will) change as your system evolves and your horizon moves.