I was begin to study the following type of stochastic optimization problem:
Let $u(k)$ and $X(k)$ are discrete random variables for all $k =1,2,...,N$ and $f$ be any concave function and $g$ be continuous function. Consider the constrained optimization problem as $$ \max_{u(k)} E[f(u(k),X(k))] $$ subject to $|u(k)| \le g(X(k))$ almost surely.
My first idea is to reformulate above as optimal control problem and try to solve it by using Pontryagin Maximum Principle (PMP) but I stuck at very beginning since PMP only works for deterministic case (and also not sure how to check the derivative conditions..) ...
I was wondering is there any recommended books/papers (which is suitable for newcomer) for studying such problem? or for its special case. Any suggestion is appreciated.
Thank you.