I was reading a differential geometry book and I came across the following claim:
Let $(M, g)$ be a Riemannian manifold and $\alpha: I \to M$ be a smooth curve in $M$ and let $\nabla: \mathcal{X}(M) \times \mathcal{X}(M) \to \mathcal{X}(M)$ be an affine connection. Then for all $t_1, t_2 \in I$, $\nabla$ induces an isomorphism between $T_{\alpha(t_1)}M $ and $T_{\alpha(t_2)}M$.
I have a few doubts:
What is this isomorphism? The author doesn't make it explicit anywhere. Is it the following?$$v \in T_{\alpha(t_1)}M \mapsto \text{parallel transport of $v$ to $T_{\alpha(t_2)}M$}$$ It's easy to show that this is a linear map. I think injectivness will be easy. About surjectiveness, let $w \in T_{\alpha(t_2)}M$ and let $\widetilde{w}$ be the parallel transport of $w$ to $T_{\alpha(t_1)}M$. Then $\widetilde{w} \mapsto w$ via the isomorphism above and we're done. Is this correct?
What's special about this particular isomorphism (whatever it is)? As finite dimensional vector spaces, any two distinct tangent spaces of $M$ are already isomorphic. So what have we gained with this?
Yes, the isomorphism is exactly the parallel transport along the path, but we should specify that we mean of "parallel transport of $v$ along $\alpha$ to $T_{\alpha(t_2)} M$. After all, parallel transport along different paths with the same endpoints typically give different isomorphisms.
Likewise, when we specify the map $\widetilde{w} \to w$ we should specify along what path from $\alpha(t_2)$ to $\alpha(t_1)$ we mean. Naturally, we should use the path $\alpha^{-1}(t) := \alpha(t_1 + t_2 - t)$ that traces out backward the image of $\alpha$. But you actually don't need that argument in this case: Since $T_{\alpha(t_1)} M$ and $T_{\alpha(t_2)} M$ both have (finite) dimension $\dim M$, linearity and injectivity imply surjectivity.
It's true that we know they're isomorphic, since they're both real vector spaces of dimension $\dim M$, and that's true even without a connection in the picture. But knowing that two vector spaces are isomorphic doesn't actually give you a correspondence between vectors in the two spaces---a particular isomorphism is precisely what does that.
One motivation for bothering with parallel transport is that we'd like to understand what curves on a Riemannian manifold $(M, g)$ play the role of straight lines in Euclidean space. A natural way to generalize straight lines is to observe that the constant-speed paths---which trace out straight lines (or just sit at a point)---are exactly the solutions to the familiar differential equation $\gamma''(t) = 0$.
On a general manifold, for a path $I \to M$ we can compute the first derivatives (tangent vectors) $\gamma'(t) \in T_{\gamma(t)} M$. But if we naively try to compute a second derivative of $\gamma$, we might write down the expression $$\lim_{h \to 0} \frac{\gamma'(t + h) - \gamma'(t)}{h}.$$ But this doesn't make sense: Typically $\gamma'(t + h)$ and $\gamma'(t)$ are vectors in two different vector spaces, $T_{\gamma(t + h)} M$ and $T_{\gamma(t)}$. If we had a choice of vector space isomorphisms $\Phi_h : T_{\gamma(t + h)} M \to T_{\gamma(t)}$ we could remedy this situation by modifying our expression for the derivative by mapping the vectors $\gamma'(t + h)$ to vectors in $T_p M$, so that we can compare them with $\gamma'(t)$. (In fact, this is precisely the reason we never had to worry about this when computing second derivatives on Euclidean space: There is a canonical isomorphism $T_p \Bbb R^n \cong \Bbb R^n$ for any $p \in \Bbb R^n$, and we can use these to define isomorphisms $T_p \Bbb R^n \cong T_q \Bbb R^n$ for any points $p, q$.) Explicitly, we can define a notion of second derivative by $$\lim_{h \to 0} \frac{\Phi_h(\gamma'(t + h)) - \gamma'(t)}{h}.$$ But this definition depends on the choice of all of the isomorphisms $\Phi_h$: Changing our minds about $\Phi_h$ changes $\Phi_h(\gamma'(t + h))$. So, in order to have a well-defined notion of second derivative, we need some way to pick isomorphisms. But this is exactly what a linear connection $\nabla$ on $M$ furnishes for us, via exactly the parallel transport map you wrote down. Explicitly, we can differentiate a vector field $Y$ at a point $p$ with respect to the vector $X$ by picking a curve $\alpha : I \to M$ with $\alpha'(0) = X$ and defining $$\nabla_X Y := \lim_{h \to 0} \frac{P_{\alpha, h}^{-1} (Y_{\alpha(h)}) - Y_p}{h}$$---here, $P_{\alpha, h} : T_p M \to T_{\alpha(h)} M$ is the parallel transport along $\alpha$ (or more precisely, $\alpha\vert_{[0, h]}$). It's a standard exercise to check that this doesn't depend on the choice of $\alpha$.
Remark To complete the story, recall that a Riemannian metric $g$ determines a preferred connection $\nabla^g$, the Levi-Civita connection. In the above definition for $\nabla_X Y$, only the values of the vector field $Y$ at points on $\alpha$ appear, so we can just as well define $\nabla^g_X Y$ for a vector field along $\alpha$---in particular the first derivative $\alpha'(t)$. So, we can define a second derivative (or acceleration) of $\alpha(t)$ by $$\nabla^g_{\alpha'(t)} \alpha'(t) ,$$ and the analogue of the equation $\alpha'(t)$ characterizing the straight lines in $\Bbb R^n$ is the equation $$\nabla^g_{\alpha'(t)} \alpha'(t) = 0 .$$ We call the solutions of this equation the geodesics of $\nabla^g$ (or of $g$). It's a straightforward exercise to unwind definitions and show that for the Euclidean metric this coincides with the usual quantity $\gamma''(t)$.