Let $u:M\times[0,T]\to N$ be a map between compact manifolds $M,N$; wlog $N\subset \Bbb R^n$.
Let $\pi_N:N_{\epsilon}\to N$ be the nearest projection map from an $\epsilon-$neighborhood of $N$ onto $N$.
How do we show that $u(x,t+\delta t) = \pi_N\left(u(x,t)+\delta t\cdot\partial_t u(x,t)\right) + o(\delta t)$?
The term $o(\delta t)$ is be understood pointwise, assuming sufficient regularity on $u$. Another question I am interested in is when $u$ is in some Sobolev space, can we show if
$u(\cdot,t+\delta t) = \pi_N\left(u(\cdot,t)+\delta t\cdot\partial_t u(\cdot,t)\right) + o(\delta t)$
where $o(\delta t)$ is understood in, says, $L^2$ sense?
The definition of the derivative tells us that $$u(x,t+\delta t) = u(x,t) + \delta t \, \partial_t u(x,t) + o(\delta t);$$ so we just need to show that the projection onto $N$ doesn't change much.
For any $y \in N_\epsilon$, the fact that $\pi_N(y)$ is a nearest-point projection tells us that the minimizing geodesic joining $y$ to $\pi_N(y)$ is orthogonal to $TN$ at $\pi_N(y)$. This (along with the fact $\pi_N|_N = \operatorname{id}_N$) tells us that along $N$, the differential $D\pi_N$ is simply the orthogonal projection onto $TN.$ Thus the definition of the derivative tells us \begin{align} \pi_N(u(x,t)+\delta t\, \partial_t u(x,t))&=u(x,t)+D\pi_N(\delta t\, \partial_t u(x,t)) + o(\delta t)\\ &= u(x,t) + \delta t\,\pi_{TN}(\partial_t u(x,t)) + o(\delta t). \end{align}
Since $u(x,t) \in N$ for all $t$, we know $\partial_t u(x,t) \in TN;$ so we can throw away the $\pi_{TN}$ and thus this is equal to $u(x,t+\delta t)$ as desired.