Let $M$ be a connected Riemannian manifold. Define $O=\{(p,v) \in TM|\, \,exp_p(v) \text{ is defined} \}$. Is $O$ an open subset of $TM$?
I know that for every point in $M$, there is a neighbourhood $V$ of $p$ and $\epsilon >0$ such that $exp_q(w)$ is defined for all $q \in V, w \in T_qM, |w| < \epsilon$ (see do-Carmo, Riemannian Geometry proposition 2.7).
However, given some $(p,v) \in O$ it is not clear if we can define the exponential at every pair in some neighbourhood of it.
The obstacle I am imagining is something like a long narrow "lacing" pointing out from a disk. Then, for a narrow range of directions (angles) we will have that the exponential map is defined for vectors of very large magnitudes, but when we change the direction a little bit, we get stuck. Of course, since I cannot make this lace to be really a line and keep the manifold non-singular, this is not really a counter-example.
As commented by Anthony, the answer is positive. The domain of the exponential map is always an open subset of $TM$. A proof can be found in Lee's book "Riemannian manifolds - introduction to curvature".
The basic idea is to realize geodesics as integral curves of a global vector field defined on $TM$ (The geodesic vector field) and then use the fact that the flow domain of a global vector field on a manifold $N$ is an open subset of $\mathbb{R} \times N$. (in this case $N=TM$).