I know that it is possible to define a covariant derivative of a tensor field given the parallel transport operator, but I wonder whether the covariant derivative of the parallel transport operator itself can be defined in any meaninful way? Or at least the covariant derivative along the curve along which the transport is happening.
I'm working on a problem where using such derivative is tempting, but I don't know if it has any sense.
First, note that the parallel transport is not an operator on $M$, neither on a curve: it is an operator from one fixed vector space to a set of sections of a vector bundle.
Given a curve $\gamma \colon I \to M$, one can define the parallel transport along $\gamma$ with basepoint $t_0\in I$. Given a point $t_0 \in I$, the parallel transport is the operator $$ P_{t_0} \colon T_{\gamma(t_0)}M \longrightarrow \Gamma\left(\gamma^*TM \right) $$ assigning to any vector $v \in T_{\gamma(t_0)}M$ the unique vector field along $\gamma$, say $V = P_{t_0}v$, such that $D_{\gamma'}V = 0$ and $V(t_0) = v$.
Also note that $\gamma$ can intersect itself for two different values $t_0 \neq t_1$, but that $P_{t_0}$ and $P_{t_1}$ are two distinct operators (if it is not clear for you, just look at the figure-eight curve).
Now, here is a question for you: how would you define for $P_{t_0}$ to be parallel? As it is not a tensor on $M$, neither along $\gamma$, this seems impossible (to me) to define this notion.