I went through the Sec I.4 of the book Controlled Markov Process and Viscosity Solutions by Fleming and Soner. In the model, $x(s)$ for $s\in[t,t_{1}]$ follows the ODE, \begin{eqnarray} x^{\prime}(s)&=&f(s,x(s),u(s)) \\ x(t)&=&x \end{eqnarray} where $f(s,x,u)$ is a continuous function on the set $[t_{0},t_{1}]\times\mathbb{R}^{m}\times U$, where $U\subseteq\mathbb{R}^{m}$ is an open set called the control space. $f$ also follows the property, \begin{eqnarray} \forall s\in[t,t_{1}], x,y\in\mathbb{R}^{n},|u|\leq\rho,~~|f(s,x,u)-f(s,y,u)|&\leq & K_{\rho}|x-y| \end{eqnarray} for some constant $K_{\rho}>0$. The control $u(\cdot)$ must be bounded and belong to the set, $\mathcal{U}^{0}(t)=L^{\infty}([t,t_{1}]; U)$, so that the IVP satisfied by $x(s)$ has a unique solution. The problem is to minimize the cost function, \begin{eqnarray} J(x,t;u)&=&\int_{t}^{t_{1}}L(s,x(s),u(s))ds+\psi(x(t_{1})) \end{eqnarray} over the set of all admissible controls $\mathcal{U}^{0}(t)$. This minimum is defined as the value function, \begin{eqnarray} V(t,x)&=&\inf_{u\in\mathcal{U}^{0}(t)}J(x,t;u). \end{eqnarray} The book states that if the control space is a compact set, $V(t,x)>-\infty$. Is it because, the $J(x,t;u)$ is somehow continuous wrt $u$ in the space $\mathcal{U}^{0}(t)$? But still, where is the compactness of $U$ getting used.
Thanks in advance.
This addresses a question in the comments as (assuming $L,\psi$ are continuous) the answer follows from the fact that the solution $x_{x_0,u}$ to the ode. is a continuous function of $x_0,u$.
A relevant result is Theorem 3 from Section XVI.1 of Kantorovich & Akilov, "Functional Analysis", 2nd. Ed: Theorem 3. If $P_y$ satisfies condition (2) for each $y \in Y_0$, with $\alpha$ independent of $y$, and if $P_y$ is continuous in $y$ at a point $y_0 \in Y_0$, then the solution of (9) depends continuously on $y$ at $y=y_0$. (The (2) means that $P_y$ is an $\alpha$ contraction, (9) means $x=P_y(x)$ and continuity means sequential continuity.) The result is remarkably straightforward to prove, it is surprising to me that it does not have the same prominence as a result as the basic Picard theorem given its utility in many situations.
Note that the Lipschitz rank $K_\rho$ is uniform as long as $\|u\|_\infty \le \rho$, and in the following I am usingthe $L^\infty$ norm (on a relevant interval).
Let $P_{(x_0,u)}(x)(\tau) = x_0+\int_t^\tau f(s,x(s),u(s))ds$. The usual Picard theorem shows that for for some $T$ (in $(t,t_1]$), if $\tau \in [t,T]$ then $P_{(x_0,u)}$ is a contraction (specifically if $K_\rho(T-\tau)<1$)and hence defines a unique solution $x_{(x_0,u)}$ on $[\tau,T]$ and this is repeated on subsequent intervals to show that there is a unique solution defined on $[t,t_1]$.
For the moment restrict attention to the interval $[t,T]$, then the above shows that if $(x_0,u)_k \to (x_0,u)$, then the solution to $x=P_{(x_0,u)}(x)$ satisfies $x_{(x_0,u)_k} \to x_{(x_0,u)}$. To extend to the whole interval, note that the initial condition on the next interval starting at $T$ is $x_{(x_0,u)}(T)$ which is continuously dependent on $(x_0,u)$ hence we see that the entire solution on $[t,t_1]$ depends continuously on $(x_0,u)$.