I am looking for any well-known method to tackle the following online problem with $T$ iterations. In each iteration $t\in\{1,2,...,T\}$, a concave objective $f_t(\cdot)$ is revealed (Hence dynamic programming does not work). $$ \max \ \left(\sum_{t=1}^{T}f_t(x_t)\right) - a\cdot\left[-Q+\sum_{t=1}^{T}x_t \right]^+\\ var. \ {\bf x}=(x_t,\ 1\le t\le T)\qquad\qquad\qquad\quad $$
where $[x]^+=\max\{0,x\}$.
The classic online convex optimization problem only has the first term, i.e., $\sum_{t=1}^{T}f_t(x_t)$. The OGD algorithm can achieve sublinear regret.
In this problem, we have a cost term, i.e., $a\cdot\left[-Q+\sum_{t=1}^{T}x_t \right]^+$, related to all the variables. Hence the OGD cannot be directly adopted here.
Any other way to tackle this?
Hint.
This problem is equivalent to
$$ \max_X\sum_k^T f_k(x_k) + \mu_j\frac{a}{\sum_k^T x_k-Q} $$
with $X = (x_1,\cdots,x_T),\ \ X_0 = (0,\cdots, 0)$ and $\mu_j$ a positively decreasing sequence. (SUMT - Fiacco & McCormick)
Another method to tackle this maximization problem is with the Dynamic Programming formulation
$$ \max_X\sum_k^T f_k(x_k) \ \ \text{s. t.}\ \ \ \sum_k^T x_k\le Q $$
or equivalently the $T$ stage process
$$ S_1(Q) = f_1(Q)\\ S_k(Q) = \max_{0\le x_k \le Q}\left[f_k(x_k)+S_{k-1}(Q-x_k)\right] $$