Ive been looking at certain economic papers, and optimal control papers. They define a state variable, $\omega$, which follows a discrete time Markov Chain. Then they define a utility function U($\omega$) on this state space, and a consumption function (both continuous and concave with first and second derivatives.) To solve the Bellman type equation:
If $\mathcal{C}$ is the space of all feasible consumption strategies:
$\underset{c \in \mathcal{C}}{\text{max}} \text{ } U(c(\omega_{0,i}))_t + \mathbb{E}[U(c(\omega_{i,j}))_{t+1}]$
subject to:
$c_{0, i}(t) + \mathbb{E}[c_{i,j} (t+1)] = \omega$
by taking derivatives with respect to $c(\omega_{0, i})$ and $c(\omega_{i,j})$. Does this make any sense at all? And if it does, does it correspond to the maximum for this bellman type equation?
Unlike your description, in economics you usually determine $c_{0,i}$ and $c_{i,j}$, i.e., consumption. Then even if $\omega$ has some discrete distribution, the choice set for a given $\omega$ would be continuous: take any $c_{0,i}\in[0,\omega]$. In that case, the first order approach would make perfect sense, given that the utility functions are concave. Now, choosing $\omega$ in the above problem seems trivial: you always choose the largest element of $\omega\in\Omega$. The real problem is deciding how to spend the wealth $\omega$ over the two periods. So you should check again what the decision variable is: I would bet is it consumption rather than wealth.
However, if the max problem is really as you write, so that $\omega$ is choice variable and discrete, then the first order approach would not make sense (as you point out), unless the optimal choice always happens to be element of the discrete set or there are some further assumptions on, say, the utility function (e.g., so that the derivative is a distance measure from the optimal point, and the lowest absolute value of the derivative would be the optimal feasible choice, as would be the case for some quadratic function). But typically, if you have to choose from finitely many alternatives, and the marginal utility at none of these alternatives is zero (that would be the optimum), then you have to compare imperfect alternatives and marginal utilities only help you in special cases.