Optimal Control - difference between indirect/direct approaches

Question

In an indirect method, my understanding is we convert the Continuous Optimal Control problem to a 2-point boundary problem by using initial conditions on the states and terminal conditions on the costates.

Then there are the direct methods based on input and/or control discretization. In direct collocation, both states and controls are discretized and a single NonLinear Program is solved.

3 questions then:

1. Is the control trajectory found through indirect/direct approaches is a locally optimal one? (I think it is using the maximum principle so I don't see why this can be a global optimal)
2. How exactly is direct collocation is different then discretizing states/controls/time?
3. Lasty, why is indirect apprach is also called "first optimize, then discretize", where is the discretization part?

Answer 1

1. solution (if one found) of the NLP in direct methods are local optimal but not neccesary global optimal (except for some special types of NLPs, e.g. convex NLP).

for the indirect approach, you solve as you already mentioned, the neccessary condition for optimality of the optimal control problem (minimum principle ,...). This forms a boundary value problem (BVP) which is often nonlinear and has several local optimal solutions (optimal state and optimal costate). The problem here is, you can solve the BVP but you obtain only ONE local optimal solution (you don't know if its global optimal). In order to find the global optimal one, you would have to determine all local optimal solutions and then use the HJB equation to check, which one of the local opt. solutions are also global optimal.

In generall, the solution depends on your initial guess (for solving numerically). For two different initial guesses you might obtain two different solutions. This holds for both methods (direct, indirect)

2. i don't understand your question! Please reformulate!

3. i am not sure why they call it like that. The main difference of direct and indirect methods are:

general optimal control: minimize a functional (performance measure) subject to constraints/bounds and of course the dynamics

direct method: solve DIRECTLY this optimal control problem as it is written there.

indirect method: solve the neccessary condition (BVP) which INDIRECTLY represents the original optimal control problem. you just rewrite the optimal control problem (which is an optimization problem) in another optimization problem. Both problems

then, you can use the same methods (e.g. collocation, shooting) for both direct and indirect methods. you only solve different sytems of ODE with different constraints.

Answer 2

1. The solution of the Euler-Lagrange equations (2-point BVP) which comes from Pontryagin's Maximum Principle (PMP) might not even be a local minimum since it constitutes a necessary condition for the input $\boldsymbol u$ to be a minimizer of $J(\boldsymbol u)$. You could also have calculated a local maximum or even a saddle point of $J(\boldsymbol u)$. In order to check if it is, in fact, a local minimizer you would have to solve the Hamilton-Jacobi-Bellman (HMB) equations which constitute a necessary and sufficient condition.
2. Collocation is just the general idea to discretize a continuous ODE in time. More specifically, in optimization, collocation means that you include each discretization step of the "collocation" scheme that you choose, e.g., Euler, Simpson, Trapezoidal, ..., into the the optimization problem as a equality constraints. This means you implicitly integrate the ODE with the chosen scheme by means of these equality constraints. This also means that it is just a synonymous way of saying you discretize the states and inputs. In my opinion this always comes along with including the discretization to your problem. This, however, is not always necessary! You can exclude the solution of the ODE from the optimization problem and only discretize the control inputs in time which yields a less complex (w.r.t. problem size) optimization problem. This approach is known as control vector parameterization (and some other names). So collocation => ODE part of problem; discretization => ODE not necessarily part of problem
3. The term direct approach comes from the fact that you directly work with the continuous optimal control problem. You discretize the integral and all time-dependent functions (ODE, constraints, etc.) in order to obtain a static nonlinear program. For this nonlinear program you can construct optimality conditions given by the KKT equations. So first you discretize the OCP, then you optimize the resulting nonlinear static program to obtain the (approximate) optimal solution of the optimal control problem. In the indirect approach, you first look at the optimality conditions of the continuous-time optimal control problem which leads you to the 2-point BVP, the Euler Lagrange equations (which, again, only constitute a necessary condition for a minimizer). This 2-point BVP is still continuous. So in order to solve it, you discretize it in time to obtain an approximate numerical solution.