Let $(M,g)$ be a Riemannian manifold; suppose for simplicity that $M$ is of Cartan-Hadamard type.
On page 4 of these notes the author states that there is a (local) correspondence between diffeomorphisms $Diff^k(M)$ and to the space of $C^k$ vector fields on $\mathbb{R}^d$ (ie: $C^k(M,M)$) given by the identification $$ X(p) = \exp_p^{-1}(g(p)) , $$ where $\exp_p$ is the Riemannian exponential map on $M$.
I'm having a bit of difficulty interpreting this expression in the case where $M=\mathbb{R}^d$.... Does it mean that given any $X:\mathbb{R}^d\rightarrow\mathbb{R}^d$ with $C^k$-components, the map $$ f^X(x)\triangleq X(x)+x, $$ is a diffeomorphism from $\mathbb{R}^d$? This interpretation seems wrong since $X(x)=x^2-x$ would clearly break the surjective requirement in $d=1$ case. So what does $\exp$ mean in that case?
Intuition: I imagine that the correspondence should somehow be given by derivatives?
I suggest you disregard this part of the notes, what's written is partly wrong or/and sloppy.
There are several places which discuss groups of diffeomorphisms as infinite-dimensional analogues of Lie groups, for instance:
Adams, Malcolm; Ratiu, Tudor; Schmid, Rudolf, The Lie group structure of diffeomorphism groups and invertible Fourier integral operators, with applications, Infinite dimensional groups with applications, Publ., Math. Sci. Res. Inst. 4, 1-69 (1985). ZBL0617.58004.
Keep in mind though, that many things you expect by analogy with the Lie groups theory do not apply. One standard mistake is to expect that every diffeomorphism close to the identity to belong to a flow generated by a vector field. This fails even for compact manifolds.
See
Palis, J., Vector fields generate few diffeomorphisms, Bull. Am. Math. Soc. 80, 503-505 (1974). ZBL0296.57008.