I understand that tangent vectors lie in separate tangent spaces based on the point on which they are tangent to a manifold, but what about vectors that are parallel transported?
For any manifold $M$, there are an infinite number of tangent spaces $T_xM$ which are defined based on the point $x$ at which the space is tangent to the manifold. I know that if a vector field is defined on a manifold, then a vector $\vec{v}$ at a point $p$ must satisfy $\vec{v} \in T_pM$. This all makes sense to me. However, I just can't wrap my head around how vectors can be parallel transported around manifolds. Wouldn't that upset the overall structure of all vectors belonging to the tangent space of a particular point?
While the tangent spaces $T_pM$ are all distinct abstract vector spaces, if $M$ has dimension $n$, then they are all isomorphic to $\mathbb{R}^n$, even though there is a priori no "canonical" isomorphism $T_p M \cong \mathbb{R}^n$. On a Riemannian manifold, parallel transport gives a way of associating to a curve $\gamma(t) \subset M$ in the manifold isomorphisms of all the tangent spaces along the curve:$$f_t: T_{\gamma(0)}M \to T_{\gamma(t)}M.$$Thus, a vector in the tangent space at the point $\gamma(0)$ can be parallel transported to a vector $f_t(v)$ in the tangent space at $\gamma(t)$.
Note that this way of identifying two tangent spaces at different points is very noncanonical. First, it depends on the choice of a Riemannian metric. Second, it may depend on the particular path chosen to join the points.