I am very new to control and mostly just reading Bellmann's stuff. He has some nice examples and writes really clearly, although there are times when his notation gets a little crazy.
Does anyone know of an example with, say, a rocket and its fuel control?
For example, suppose a rocket is given a certain amount of fuel and the pilot wants to use the fuel so that he runs out just as he lands. The amount of fuel can be whatever to start and constant velocity may not get rid of all the fuel. Having left over fuel can be viewed as just as bad as not having enough fuel so the issue of crashing doesn't come into the problem. For example, it could have been an initial amount of water out of the pilot's thermos and he has to drink continuously so that he finishes the water exactly as they land. Thanks.
It sounds to me like you don't have a well enough defined problem. When you start getting into Bellman and Pontryagin's work, you're getting into a specialization of control theory known as optimal control. With your specific problem, a rocket may land while expending all of its fuel in any number of ways. That is, there are an infinite number of paths it may take. Optimal control usually only works under the assumption that there exists 1 locally optimal solution. Refine the problem with a performance index such as minimum time to land. You can then include expending all fuel as a terminal constraint.
On the other hand, if you do not want to include any additional performance measurements, you can solve the problem by picking any of the infinite paths available. To do this, you won't want to use Bellman or Pontryagin.