A question about the lemma in the title:
Lemma 4.1. For any $p \in M$ there exists a number $c > 0$ such that any geodesic in $M$ that is tangent at $q \in M$ to the geodesic sphere $S_r(p)$ of radius $r < c$ stays out of the geodesic ball $B_r(p)$ for some neighborhood of $q$.
I'll write down the proof and write the question after
Proof. Let $W$ be a totally normal neighborhood of $p$. Using the lemma of homogeneity, we can suppose, by conveniently restricting the the interval of definition, that all of the geodesics of $W$ have velocity one. We can, therefore, restrict ourselves to the unit tangent bundle $T_1 W$ given by $$ T_1 W = \left\{(q,v) : q \in W, v \in T_q M, |v| = 1 \right\} $$ Let $\gamma : I \times T_1 W \to M$, $I = (-\epsilon,\epsilon)$, be the differentiable mapping such that $t \to \gamma(t,q,v)$ is the geodesic that at the instant $t=0$ passes through $q$ with velocity $v, |v| = 1$. Define $u(t,q,v) = \exp_p^{-1}(\gamma(t,q,v))$ and $$ F:I\times T_qW \to \mathbb{R}, \;\;\;\; F(t,q,v) = |u(t,q,v)|^2. $$ $F$ measures the square of the "distance" from $p$ to a point that is moving along the geodesic $\gamma$. It is clear that $u$ and $F$ are differentiable, and that $$ \begin{array}{l} \frac{\partial F}{\partial t} = 2 \left\langle \frac{\partial u}{\partial t}, u\right\rangle \\ \frac{\partial^2 F}{\partial t^2} = 2 \left\langle \frac{\partial^2 u}{\partial t^2}, u\right\rangle + 2 \left| \frac{\partial u}{\partial t} \right|^2 \end{array} $$ Now let $r > 0$ be chosen so that $$ \exp_p B_r(0) = B_r(p) \subset W $$ If a geodesic $\gamma$ is tangent to the geodesic sphere $S_r(p)$ at the point $q = \gamma(0,q,v)$, then, from the Gauss lemma $$ \left\langle \frac{\partial u}{\partial t}(0,q,v), u(0,q,v) \right\rangle = 0 $$
Question : How is the Gauss lemma exactly applied here?
Let us write it out in detail. We begin with a few observations:
$\displaystyle\frac{\partial \gamma}{\partial t}(0,q,v)$ is the velocity of the geodesic that at the instant $t = 0$ passes through $q$ with velocity $v$, which is to say that $\displaystyle\frac{\partial \gamma}{\partial t}(0,q,v) = v$ is a tangent vector based at $\gamma(0,q,v) = q$.
Let us write $\tilde v = u(0,q,v) = \exp_p^{-1}\big(\gamma(0,q,v)\big) = \exp_p^{-1}(q)$. Then $\tilde v$ is a tangent vector based at $p$, and if we assume, as do Carmo does, that $q\in S_r(p)$, and all geodesics under consideration have unit speed, then by our definitions, $\tilde v = \tilde \gamma'(r)$ for an appropriate choice of geodesic $\tilde\gamma$.
By the last two points and the chain rule, $\displaystyle\frac{\partial u}{\partial t}(0,q,v) = \big(d \exp_p^{-1}\big)_{q}(v) = \big( (d \exp_p)_\tilde v\big)^{-1}(v)$ is a tangent vector based at $p$.
Now, we can write \begin{align*} \left<\frac{\partial u}{\partial t}(0,q,v),u(0,q,v)\right>_p &= \left<\big( (d \exp_p)_\tilde v\big)^{-1}(v),\tilde v\right>_p \\ &= \left<\big(d \exp_p\big)_\tilde v\big( (d \exp_p)_{\tilde v}\big)^{-1}(v),\big(d\exp_p\big)_{\tilde v}(\tilde v)\right>_q \\ &= \left<v,\big(d\exp_p\big)_{\tilde v}(\tilde v)\right>_q\\ &= \left<v,\tilde \gamma'(r)\right>_q, \end{align*} where we applied Gauss' lemma (p. 69 of do Carmo) on the second line. (Implicitly we have used the common identification, as do Carmo does in his proof of Gauss' lemma, that $T_qM\approx T_\tilde vT_qM$.) Our assumption that $v$ be tangent to the geodesic sphere $S_r(p)$ tells us that the last expression in the display above is $0$.