Let $(M,g)$ be a Riemannian manifold, compact if need be. Take an arbitrary $(p,v)\in TM$ and consider the geodesic starting at $p$ in direction $v$, i.e., $\gamma\colon I\to M, t\mapsto \exp_p(tv)$.
Is it true that \begin{align*} \operatorname{dist}_g(p, \exp_p(tv)) = t|v|_g \end{align*} for all $t\in I$? How about t=1?
I've tried calculating the length of $\gamma$ via $\gamma'(t) = d(\exp_p)_{tv}(v)$ but couldn't arrive anywhere.
On
Since for any geodesic $\alpha$, $0=\frac{D}{dt}\alpha'(t)=\frac{1}{2}\frac{d}{dt}\langle \alpha'(t), \alpha'(t) \rangle$, it turns out that $|\alpha'(t)|=|\alpha'(0)|$, $\forall t$.
It is also proved in do Carmo's book page 70 that, inside geodesic balls $B(p,\epsilon)$ centered at $p$, radial geodesics are the curves that minimize the Riemannian distance ( for this you need Gauss Lemma page 69, and there is no way around it). Since $\gamma(t)=\exp_p(tv), |t|<\epsilon$ is the radial geodesic from $p$ to $\exp_p(t_0v)$, $\forall$ $|t_0|<\epsilon$:
$d(p,\exp_p(t_0 v))=\int_0^{t_0}|\gamma'(t)|dt=\int_0^{t_0}|\gamma'(0)|dt=t_0|v|$, since by the very definition of the exponential map, $(t\mapsto \exp_p(tv))'(0)=v$. This is valid for any $v\in T_pM$, not just for those with $|v|=1$.
Assuming that $|v|=1$, the formula \begin{align*} \operatorname{dist}_g(p, \exp_p(tv)) = t|v|_g \end{align*} holds for all $t$ in a sufficiently small interval $[-\epsilon, \epsilon]$. You will find a proof in any book on Riemannian geometry; my favorite reference is do Carmo, "Riemannian Geometry", Proposition 3.6. But for large $t$ this identify typically fails, for instance, think about the case $M=S^1$. However, the identity holds for all $t$ in some important special cases such as when $M$ is a Hadamard manifold, see Chapter 7 of do Carmo's book.