Consider a vector bundle $E \to M$. Given a connection $\nabla$, it induces a parallel transport, which (in particular) is a choice of isomorphism $T_{\gamma} : E_{\gamma(0)} \to E_{\gamma(1)}$ for each path $\gamma$, respecting path concatenation (i.e. it becomes composition of the parallel transports).
We also know that it's possible to recover what the connection $\nabla$ was just from knowing these parallel transport operators, by taking a derivative-like limit. What I want to know is what are the conditions on a family of isomorphisms as described above to have come from a connection. I feel like just the above should be nearly enough, but there should also be some condition saying that this choice of operator "varies smoothly". However, I can't figure out what the proper formalization of that should be.