What is infinite horizon problem?

Question

In optimal control, What is infinite horizon problem? What is the difference between finite and infinite horizon? What are their real life examples (finite & infinite)?

Answer 1

When we talk about controlling a system, the term "optimal control" makes sense only if we specify, with respect to what condition we define optimality. That is 1) does achieving some values for the system states in minimum time is highly desired? 2) or are you concerned about the energy spend to attain certain values for system states? 3) or being conservative in terms of the time and energy spend means a lot to you. These are usually called performance measures (cost function).

And next question is about the horizon...or the time span of the system operation during which you are concerned about such defined performance measures. If you want to control the system, meeting the performance measures for a finite time say $T$, then the problem is finite horizon and if you are concerned about the optimality during the whole time span i.e till $t=\infty$, then it is an infinite horizon problem.

The problem of deriving control $u(t)$, $t=[0,T]$ for the system \begin{align} \dot{x}(t)=Ax(t)+Bu(t) \end{align} such that the performance index \begin{align} PM=\int_0^T x(t)'Qx(t)+u'(t)Ru(t) {\rm d}t \end{align} is minimised is a finite horizon problem

The problem of deriving control $u(t)$, $t=[0,\infty]$ for the system \begin{align} \dot{x}(t)=Ax(t)+Bu(t) \end{align} such that the performance index \begin{align} PM=\int_0^\infty x(t)'Qx(t)+u'(t)Ru(t) {\rm d}t \end{align} is minimised is an infinite horizon problem

Answer 2

Consider a discrete time dynamical system, that evolves in time at discrete time steps. If such a discrete time evolving process is embedded with

a. Cost that is additive over time
b. A control, or decision variable, that is externally selected

we might come up with an optimization problem, otherwise known as an optimal control problem.

In such a problem. the horizon is the number of times a control is applied, or decisions need to be made. Let us call such a horizon as H.

1. If such a horizon is finite, we refer to the problem as a finite horizon problem
2. Horizon is stochastic or random, and for any instance of the problem is finite. But given any N, one can find a version of the problem which has a horizon greater than N. Such problems fall under stochastic shortest path problems. Some authors consider this class to fall under the infinite horizon setup
3. Horizon is infinite. We consider ongoing problems which don't necessarily terminate under such a setup. One is forced to use a value function that is the discounted sum of costs, or average costs under this setup. Simply using total cost over the stages would lead to non-finite value.

Example of finite horizon:

1. Games: Any games which terminate in finite time like tic-tac-toe
2. Any discrete optimization problems over a fixed time. Examples:
   • Inventory control over fixed time periods
   • Picking m stocks over n time periods

Some examples of stochastic horizon from the area of game play:

1. Chess: Most games terminate when some player wins, but given any number of moves, one can find a game which does not terminate

Some examples of infinite horizon from the area of marketing:

1. Ads to show: Customers are shown ads by google depending on their state (say loyalty). One may assume (for practical purposes) that for such a process, the number of ads shown over a lifetime is not bounded.

Reference: Dynamic Programming and Optimal Control by Dimitri Bertsekas