Let $M$ be a Riemannian manifold and $\exp_x:U_x\subset T_xM\to O_x\subset M$ denote the exponential map at $x$ such that it is a diffeomorphism between $U_x$ and $O_x$. Define $U:=\cup_xU_x\subset TM$ and $O:=\cup_x O_x\subset M$, and let $\exp:U\to O$ be the exponential map defined as $\exp(v)=\exp_x(v)$ if $v\in U_x$.
Is there a way to compute or differentiating $\exp$ with respect to $x$?
Even a simple example in a simple space ($S^2$ for instance) is very welcome.
It depends on what you mean by "compute".
Let me work out your example of $S^2 = \{(a, b, c)\in\mathbb{R}^3:a^2 + b^2 + c^2 = 1\}$ and the point $x = (1, 0, 0)$.
The tangent space of $S^2$ at $x$ is the plane orthogonal to the $a$-axis, or simply the plane $a = 0$. Thus a tangent vector is of the form $(0, r\cos\theta, r\sin\theta)$.
The geodesics on $S^2$ are the great circles. So imagin you are an ant standing on $S^2$ at the point $x$, and you want to go in the direction of $(0, r\cos\theta, r\sin\theta)$, then you are actually moving along the great circle $\{(\cos\varphi, \sin\varphi\cos\theta, \sin\varphi\sin\theta): \varphi\in[0, 2\pi)\}$. Drawing a picture is recommended here.
Hence if you go along that geodesic for the same length of your tangent vector $(0, r\cos\theta, r\sin\theta)$, which is $r$, then you arrive at $(\cos r, \sin r\cos\theta, \sin r\sin\theta)$.
This means that $\exp_x(0, r\cos\theta, r\sin\theta) = (\cos r, \sin r\cos\theta, \sin r\sin\theta)$.
As you can see, even for such a simple example, the explicit formula is rather complicated. Usually people don't compute things so explicitly, but use certain properties to characterize the result.
After posting the answer, I realized that I didn't mention anything about differentiating $\exp$. But this is almost tautologically equal to the identity map on $T_xM$.