I am trying to solve Exercise 25.4 of Tomas Björk's Arbitrage Theory in Continuous Time.
The exercise goes as follows:
Consider the problem of minimizing
$$ \mathbb{E}\left[\int_0^T F(t, X_t^u, u_t)dt + \Phi(X_T^u)\right] $$ subject to
$$ dX_t = \mu(t, X_t, u_t)dt + \sigma(t, X_t, u_t)dW_t $$ and $u(t, x) \in U$. Here $u$ is some control law, $U$ is some constraint on the control law, $\Phi$ is some terminal cost function and $F$ is a running cost function. Define the total cost process $C(t, u)$ by
$$ C(t, u) = \int_0^t F(s, X_s^u, u_s)ds + \mathbb{E}_{t, X_t^u}\left[\int_t^T F(t, X_t^u, u_t)dt + \Phi(X_T^u)\right].\tag{1} $$ Then, I am to show that
(a) If $u$ is an arbitrary control law, then $C$ is a submartingale.
(b) If $u$ is an optimal control law, then $C$ is a martingale.
I already got confused in part (a), as I seem to be getting that $C$ is a martingale even for an arbitrary $u$. To do this, I take conditional expectations of (1), split the first integral according to the smaller information set, use law of iterated expectations on the second integral, split the inner integral again according to the information set. Things then cancel out and I get the martingale. (If you would like me to elaborate, please ask. Seeing as it is obviously wrong, I didn't bother)
Obviously, I somehow need to use the (sub-)optimality of $u$, but I'm not seeing how. Any help would be greatly appreciated, thanks!
As the commenters have suggested, there seems to be an error in this exercise. Suppose instead of equation ($1$), $C(t, u)$ was defined as
$$ C(t, u) = \int_0^t F(s, X_s^u, u_s)ds + V(t, X_t^u), $$ where $V(t, X_t^u)$ is the optimal value function. In other words, it is the last term in (1) with a $\min_{u \in U}$ prefix. Then, by Ito's Lemma
\begin{align} dC_t &= F(t, X_t^u, u_t)dt + \frac{\partial V}{\partial t}(t, X_t^u) + dV_t\\ &= \left\{F(t, X_t^u, u_t)dt + \frac{\partial V}{\partial t}(t, X_t^u) + \mathcal{A}^uV(t, X_t^u)\right\}dt + \sigma (t, X_t^u, u_t)\frac{\partial V}{\partial X_t^u} dW_t, \end{align} where $\mathcal{A}^u$ is the infinitesimal generator. The term in brackets is larger zero for an arbitrary control law and equal to zero for the optimal control by the HJB equation. The result then follows directly.