Let $f\in C^\infty(M,\mathbb R)$ for some smooth manifold $M$, and consider a vector field $X\in\Gamma(TM)$ and a point $p\in M$.
The Lie derivative ${\cal L}_X f$ of $f$ along $X$ is usually defined (e.g. on Wikipedia or in these notes) via the flow of the vector field, with something like $$(\mathcal L_X f)(p)\equiv \lim_{t\to 0}\frac{f(P(t,p))-f(p)}{t},$$ where $P(0,p)=p$ and $\partial_t P(t,p)=X(P(t,p))$.
I've seen this definition used in a few different sources. What I'm wondering is: isn't this equivalent to what we get taking the differential of $f$ at the point along the direction given by the vector field, i.e. $df_p(X(p))$? If so, is there an advantage in using one definition over the other, or it is just a matter of notational convenience?
The Wikipedia page also remarks that "Setting $\mathcal L_X f=\nabla_X f$ identifies the Lie derivative of a function with the directional derivative", which again makes me wonder whether I can generally say that $(\mathcal L_X f)(p)=df_p(X(p))$, or whether I'm missing something in the notation.
Yes, if you start with the pull-back definition, then by definition of a flow (i.e differentiating gives the corresponding vector field), it follows from the chain rule that $(\mathcal{L}_Xf)(p)=df_p(X_p)$. So, of course, what is a theorem vs what's a definition is entirely a matter of preference in the order of presentation.
Personally, I like the definition of Lie-derivatives using the flow because the definition stays the same for tensor fields of any type, and it literally says that it's a measure of how much a tensor field changes along a vector field's flow. \begin{align} (\mathcal{L}_XT)(p):= \dfrac{d}{dt}\bigg|_{t=0}(\Phi_t^*T)(p)= \lim_{h\to 0}\frac{(\Phi_t^*T)(p)-T(p)}{t}, \end{align} where $\Phi$ is the flow of the vector field $X$. Geometrically, I love this definition. But this is tough to work with computationally. Even worse: on first glance, it's not clear the limit even exists, let alone the result being a smooth tensor field. Once we get past all of this though, everything becomes smooth sailing (most of the time, I would rather have a simple intuitive definition, and work hard to get theorems than the other way around).
Of course, you can also proceed more algebraically, by defining $\mathcal{L}_Xf=df(X)$, and then defining its action on vector fields and one-forms, and then extending to arbitrary tensor fields by requiring the Leibniz rule etc. One can show that if you have an operation defined on functions and vector fields, then you can extend it to the full tensor algebra by requiring it to commute with contractions and requiring the product rule. This is a very nice theorem and it easily defines for us the Lie-derivative of arbitrary tensor fields (we can do similar things for the covariant derivative), but I feel that algebraic slickness hides the geometry, hence I'm not a huge fan (though I certainly appreciate it).