I'm not really having a good background in math, so please correct me if I say something very vague or even wrong.
Suppose a function $\rho=f(\theta):\mathbb{R}\to\mathcal{W}_p(\mathbb{R}^d)$. I would like to calculate a sort of derivative $\tilde{\frac{d}{d\theta}}\rho=f'(\theta)$, which magnitude corresponds to $$ \left\vert \tilde{\frac{d}{d\theta}}\rho \right\vert =\lim_{\Delta\theta\to0}\frac{\mathcal{W}_p[f(\theta+\Delta\theta),f(\theta)]}{\Delta\theta}. $$ Although, at this point I'm not at all clear about (1)what kind of quantity $\tilde{\frac{d}{d\theta}}\rho$ is, and (2)what kind of norm $\vert\cdot\vert$ we use here. For example a common point-wise derivative $$ \frac{d\rho}{d\theta}(x) = \lim_{\Delta\theta\to0}\frac{\rho(x;\theta+\Delta\theta) - \rho(x;\theta)}{\Delta\theta} $$ with its $L_2$-norm definitely does not correspond to such quantity.
So the question is, is there any concept that corresponds to or similar at all to this kind of derivative?
Th metric derivative in $W_p(\mathbb{R}^d)$ can indeed be interpreted as the norm of a derivative. In the case $p=2$ this is known as the formal Riemannian structure on $W_2(\mathbb{R}^d)$ due to Otto. It is not so easily made rigorous (that's one reason to work with the metric derivative instead), so I'll stay on the formal level here.
The key is the Benamou--Brenier formula $$ W_2(\rho_0,\rho_1)^2=\inf\left\lbrace \int_0^1 \int \lvert v_t\rvert^2\,d\rho_t\,dt: \dot\rho_t+\mathrm{div}(\rho_t v_t)=0\right\rbrace, $$ which looks suspiciously like the formula for the length distance on a Riemannian manifold.
Let $(\rho_t)$ be a (sufficiently regular) curve in $W_2(\mathbb{R}^d)$ with $\rho_0=\rho$ and $\dot\rho_0=\xi$. There exists a unique (under appropriate restrictions) curl-free vector field $v$ such that $$ \xi+\mathrm{div}(\rho v)=0. $$ This establishes an isomorphism $K_\rho$ between the tangent vectors at $\rho$ and the curl-free vector fields. We can then define a Riemannian metric on $W_2(\mathbb{R}^d)$ by $$ g_\rho(\xi,\eta)=\int K_\rho(\xi)\cdot K_\rho(\eta)\,d\rho. $$ The associated length metric is exactly the Wasserstein metric $W_2$.
For small $h$, the distance $W_2(\rho_t,\rho_{t+h})$ is approximately the length of the curve $(\rho_s)_{s\in[t,t+h]}$. Thus $$ \frac{W_2(\rho_t,\rho_{t+h})}{h}\approx \frac 1 h\int_t^{t+h}\left(\int \lvert K_{\rho_s}(\dot\rho_s)\rvert^2\,d\rho_s\right)^{1/2}\,ds\to \left(\int \lvert K_{\rho_t}(\dot\rho_t)\rvert^2\,d\rho_s\right)^{1/2}, $$ which is exactly the norm of the tangent vector $\dot\rho_t$ in $(T_{\rho_t}W_2(\mathbb{R}^d),g_{\rho_t})$.
For $p\neq 2$ there is also a Benamou--Brenier-type formula (replace the exponent $2$ by $p$). In this case you get a formal Finsler structure instead of a Riemannian one, but the rest is quite similar.