Let $\gamma:I\to M$ be a smooth curve (self-intersection is allowed) and $V:I\to TM$ a smooth vector field along $\gamma$. The covariant derivative of $V$ along $\gamma$ is denoted by $D_tV$, we have $$ D_tV=\nabla_{\dot\gamma(t)}\widetilde V $$ if $V$ is a restriction of a vector field $\widetilde V$, $\nabla_X$ is the usual covariant derivative.
We know that $D_t$ induces the parallel transport on $\gamma$, i.e. $$ P_{t_0t_1}:T_{\gamma(t_0)}M \to T_{\gamma(t_1)}M $$ which is a linear isomorphism. Conversely, we may recover $D_t$ from $P$ by
$$D_{t}V(t_0)=\lim_{t\to t_0} \frac{P_{t_0t}^{-1}V(t)-V(t_0)}{t-t_0} $$
I tried to prove this by using parallel frame along $\gamma$, i.e. let $\{E_1,\dots,E_m\}\subset T_{\gamma(t_0)} $ be a basis of $T_{\gamma(t_0)}$ and define $$ E_i(t):=P_{t_0t}E_i. $$ Since $P_{t_0t}$ is a linear isomorphism, $E_i(t)$ forms a basis for $T_{\gamma(t)}$ hence we may write $$ V(t)=V^i(t)E_i(t). $$
Direct computation shows that $$ \lim_{t\to t_0} \frac{P_{t_0t}^{-1}V(t)-V(t_0)}{t-t_0}=\dot V^i(t_0)E_i, $$ assuming $V^i$ is differentiable. On the other hand, we also have $$\begin{align} D_{t}V(t_0) &=\dot V^i(t_0)E_i(t_0) + V^i(t_0)D_t E_i(t_0) \\ &= \dot V^i(t_0)E_i \end{align}$$ since $D_t E_i(t)=0$, which shows that these expressions are equal.
However, I cannot justify a priori why should $V^i(t)$ be smooth (or differentiable).
The definition of $V$ being smooth merely say that its coordinate functions are smooth in any chart of $TM$, but $E_i(t)$ need not come from coordinate chart.
Could anyone please help me to complete my proof or suggest another correct proof to this?
If you parallel transport a tangent vector $v\in T_{\gamma(0)} M$ along a curve $\gamma$, then the obtained vector field $V(t)$ is automatically smooth in $t$. Hence in your case, the frame fields $E_i(t)$ are smooth. You can start with an orthonormal basis $\{E_1,\dots,E_n\}$, then since parallel transport preserves the metric $g$, also the $E_i(t)$ will be orthonormal and you can express $V^i(t)$ as $V^i(t)=g(E_i(t),V(t))$, hence they are smooth as well.
Now why does parallel transport automatically produce smooth fields? Looking at the corresponding proposition in do Carmos book (Prop. 2.6) tells us that the coordinates $v^k(t)$ (with respect to a smooth frame) of the vector field $V(t)$ that we obtain by parallel transporting $v$ satisfy an ODE of the form: $$ \dot v^k(t) + \sum_{l}v^l(t)a^k_l(t) = 0, $$ where the matrix field $a^k_l$ is an expression in the coordinate functions and the Christoffel symbols, hence smooth. Now you can use the following bootstrapping argument: You already know from ODE theory that the $v^k$ are $C^1$, use this to start an induction argument: If the $v^k$ are also $C^m$, then $\dot v^k = - \sum_{l}v^l(t)a^k_l(t) \in C^m$, hence $v^k \in C^{m+1}$.