Together with some friend we are making exercises containing the Bellman equation and we faced a pretty difficult one:
$x_{k+1} = x_k + u_k$ for $k = 0,1,2,3$, with starting state $x_0 = 5$ and cost function $\sum_{k=0}^3(x_k^2 + u_k^2)$.
Minimise the cost function when the restriction on $u_k$ is given by $ u_k \in$ {$u \in \mathbb{Z}: 0 \leq x_k + u \leq 5$}.
We tried using the following recursive Bellman Equation:
$V(k+1, x_{k+1}) = \inf x_{k+1}^2 + u_{k+1}^2 + V(k,x_k)$
When we tried using this formula we got for every $x = 5$ and for every $u = 0$ but this can't be correct...
Who can help us out?