I am studying differential geometry and I recently learned about affine connections. My understanding of them so far is that they are a generalization of the directional derivative to arbitrary manifolds and allows us to differentiate a vector field in the direction of another vector field. They are needed as we otherwise cannot compare tangent vectors that belong to different tangent spaces. As a consequence, the covariant derivative thus lets us "connects" various tangent spaces together.
With this in mind, I am confused on what new idea the covariant derivative is trying to capture. It seems that is is also differentiating a vector field with respect to another vector field (or a vector field that is defined using a curve). I am having a hard time understanding what the covariant derivative is really doing that an affine connection is not already doing. What does it intuitively mean for a covariant derivative to be compatible with an affine connection?