I am trying to study stochastic control from the 2009 book "Continuous-time Stochastic Control and Optimization with Financial Applications" by Huyen Pham.
I am in chapter 3, working through the derivation of the Hamilton-Jacobi-Bellman equation. The notation so far introduced is the following:
$v(t,x)$ is the value function if the process starts from $x$ at time $t$.
$f(t,x,a)$ is the reward function at time $t$, at "position" $x$ if action $a$ is picked.
$\mathcal{L}^a$ is the infinitesimal generator of $X_t$ for the constant control $a$.
The author says: by the principle of optimality, we obtain
$$ v(t,x) = \mathbb{E} \left[\int_{t}^{t+h} f(X_u^{t,x},u,a)du + v(t+h,X^{x,t}_{t+h})\right] $$
Then, he uses Ito's Lemma to find that
$$v(t+h,X^{x,t}_{t+h}) = v(t,x) + \int_t^{t+h}\left(\frac{\partial v}{\partial t}+\mathcal{L}^a \right)(s, X_u^{t,x})du + (local) \ Martingale$$
and he concludes by saying to substitute $v(t+h,X^{x,t}_{t+h})$ to find
$$0 \geq \mathbb{E}\left[\int_{t}^{t+h} \left(f(X^{t,x}_u,a,u) + \left(\frac{\partial v}{\partial t}+\mathcal{L}^a\right) (s, X_u^{t,x})du\right) \right] $$
What I do not understand is how he manages to ignore the expected value of the local martingale. It doesn't seem like we are assuming the last component of to be an actual martingale: I don't see how that would follow by the assumptions, a belief that is made stronger by the fact that the author himself writes "local" Martingale.
Can anyone help me?
An observation: here we really want that expected value to be 0, since in later parts a very similar derivation is used but with equality instead of $\leq$, meaning that if the expected value of the local martingale is not zero, equality cannot hold.
I think the local martingale part is :
$$LM_t = \int_{t}^{t+h} \hspace{0.1cm} D_xv(s, X_s^{t,x}) \hspace{0.1cm} \sigma(X_s^{t,x}, a) dW_s$$
I assumed that $v$ is $C^{1,2}$ so that $D_xv$ is $C^{1}$. So you can use continuity (notice that there is lipschitz assumption on $\sigma$) argument on the compact $[t, t+h]$ to show that $LM$ is a true martingale and then $\mathbb{E}(LM_t) = 0$