$$ \dot x(t)=Ax(t)+Bu(t)\\ x(t_0)=x_0\\ x(t_f)=x_f\\ a\leq x\leq b $$ $x(t)$ is driven by $u(t)$ to satisfy the above differential equation, two boundary conditions, and the inequality. Here, $x_0$ and $x_f\in[a,b]$, $t\in[t_0,t_f]$.
My question:
What condition shall I impose on the above formulation to guarantee the feasible set of $u$, i.e. $\mathcal U(u)$ is convex in $u$?
Thanks in advance!
You don't need any special conditions, it is already convex. To see that we can write the solution of the differential equation as
$$ x_f = x(t_f) = e^{A(t_f - t_0)} x_0 + \int_{t_0}^{t_f} e^{A(t_f - \tau)} B u(\tau) d \tau $$
Now suppose $u_1$ and $u_2$ are two functions that satisfy the above equation, i.e. $u_1, u_2 \in \mathcal{U}(u)$. We need to show that $\alpha_1 u_1 + \alpha_2 u_2$ also satisfies it where $\alpha_1 + \alpha_2 = 1$, $\alpha_1, \alpha_2 > 0$.
$$\begin{align} \alpha_1 x_f &= \alpha_1 e^{A(t_f - t_0)} x_0 + \int_{t_0}^{t_f} e^{A(t_f - \tau)} B [\alpha_1 u_1(\tau)] d \tau \\ \alpha_2 x_f &= \alpha_2 e^{A(t_f - t_0)} x_0 + \int_{t_0}^{t_f} e^{A(t_f - \tau)} B [\alpha_2 u_2(\tau)] d \tau \\ x_f &= e^{A(t_f - t_0)} x_0 + \int_{t_0}^{t_f} e^{A(t_f - \tau)} B [\alpha_1 u_1(\tau) + \alpha_2 u_2(\tau)] d \tau \end{align}$$
This can easily be generalized to $n$ dimensions.