Let $(M,g)$ be a Riemannian manifold with induced metric $d$ and injectivity radius $r>0$. Let $p, q$ be two points in $M$ such that $d(p,q)<r$. It is easy to see that $p$ and $q$ can be connected by an unique minimizing geodesic $\gamma=\{\gamma(t):0\le t\le 1 \}$ with $\gamma(0)=p$ and $\gamma(1)=q$. Denote by $\Gamma_{p\to q}:T_p(M)\to T_q(M)$ the parallel transport along the geodesic $\gamma$ from $p$ to $q$.
My question is: for a tangent vector $X\in T_p(M)$, does the following holds $$\exp_q^{-1}(\exp_p(tX)) = \exp_q^{-1}(p)+t\Gamma_{p\to q}(X)+O(t^2), \quad t\to 0?$$
Any comments or hints will be appreciated. TIA.
It does not hold : On $2$-dimensional unit sphere, consider two points of $\frac{\pi}{2}$ distance. And parallel vector field on the geodesic between two points is perpendicular to the geodesic (cf. Parallel transportion for Alexandrov space with curvature bounded below - Petrunin) :
When $M=\mathbb{S}^2,\ |p-q|=\frac{\pi}{2}$, then $c$ is unit speed geodesic from $p$ to $q$. Assume that $X(t)$ is a unit parallel vector field along $c(t)$ s.t. $X(0)\perp c'(0)$.
Then $$A=0,\ B=\exp_p^{-1}\ q,\ C=\exp_p^{-1}\ \exp_q\ tX(\pi/2)$$ is a triangle in $T_pM$ s.t. $\angle\ BAC = t$.
When $A,\ B,\ C'$ is triangle with $\angle\ ABC'=\pi/2$ and $|B-C'|=t$, then we find $|C-C'|$ : By cosine law, \begin{align*} |C-C'|^2&=t^2+(\pi\ \sin\ \frac{t}{2})^2-2t(\pi \ \sin\ \frac{t}{2})\cos\ \frac{t}{2} \\& =( t-\pi\ \sin\ \frac{t}{2})^2+ O(t^4) \\ |C-C'| &=t-\pi\ \sin\ \frac{t}{2}+ O(t^2)\end{align*}