Given $L:\mathbb{R}\times \mathcal{P}(\mathbb{R}^n) \to \mathbb{R}$, where $\mathcal{P}(\mathbb{R}^n)$ is the space of probability densities on $\mathbb{R}^n$. I want to calculate
$$ \frac{d}{d\epsilon}\Big|_{\epsilon \to 0}L(\rho(x)+\alpha(x),\rho+\epsilon \alpha ), $$ where $\alpha=\tilde{\rho}-\rho$, and $\tilde{\rho},\rho\in \mathcal{P}(\mathbb{R}^n)$.
Is my calculation correct : $$ \frac{d}{d\epsilon}\Big|_{\epsilon \to 0}L(\rho(x)+\epsilon\alpha(x),\rho+\epsilon \alpha )=\alpha(x) L'(\rho(x),\rho)+dL(\rho(x),\rho;\alpha)$$ with $dL(\rho(x),\rho;\alpha)$ the Gateaux derivative?
Referring to your original (now deleted) post, I will assume that you're trying to compute the first variation of a functional $\mathscr F$ over the space of probability measures, as defined in chapter 7 of Santambrogio's Optimal Transport for Applied Mathematicians.
First thing to note is that in that case, $\mathscr F$ is actually defined for absolutely continous measures by the expression $$\mathscr F(\mu) := \int_{\mathbb R^n} L(\rho(x),\rho)\ d\lambda(x) $$ Where $\lambda$ is the Lebesgue measure and $\rho:=\frac{d\mu}{d\lambda} $ is the Radon-Nikodym derivative of $\mu$ with respect to $\lambda$. Hence for the above expression to make sense, we need $L$ to be defined on $X := \mathbb R\times L^1(\mathbb R^n) $, where $L^1(\mathbb R^n) $ is the space of measurable and integrable functions on $\mathbb R^n$ (that is because any probability density is automatically integrable by definition). Also I took the domain to be $\mathbb R^n$ to fix ideas, but you can clearly replace it by any suitable $\Omega$ you wish.
Now to compute the first variation of $\mathscr F$, you can let $\theta$ be any density function on $\mathbb R^n $ (or any $L^1$ function really) and let $\chi := \theta\cdot\lambda $, so that for any $\varepsilon >0$
$$\mathscr F(\mu +\varepsilon \chi) - \mathscr F(\mu) = \int_{\mathbb R^n} \big[L(\rho(x) + \varepsilon\theta(x), \rho + \varepsilon\theta) - L(\rho(x),\rho)\big]\ d\lambda(x) $$
Hence we can see that under suitable regularity conditions, the first variation of $\mathscr F$ in direction $\chi$ will be given by $$\frac{\delta \mathscr F}{\delta \mu}(\mu) : x\in\mathbb R^n\mapsto dL(\rho(x),\rho;\tilde\theta(x)) $$ Where $\tilde \theta(x) := (\theta(x),\theta)\in X$ and $dL(\rho(x),\rho;\tilde \theta(x))$ denotes the Gâteaux derivative of $L$ at $(\rho(x),x)\in X$ in direction $\tilde \theta(x)$.