Definition: Let $f$ be a differentiable real-valued function on $\mathbb{R}^3$, and let $\mathbf{v}_p$ be a tangent vector to it. Then the following number is the derivative of a function w.r.t. the tangent vector
$$ \mathbf{v}_p[f]=\frac{d}{dt}(f(\mathbf{p}+t \mathbf{v}))|t=0 $$ Further, there is this
Lemma: If $\mathbf{v}_p=(v_1,v_2,v_3)_p$ is a tangent vector to $\mathbb{R}^3$, then
$$\mathbf{v}_p[f]=\sum_i^3 v_i \frac{df}{dx_i} (\mathbf{p})$$
Ques: How does this gradient term $\frac{df}{dx_i} $ come into the picture? Is it that in the relation $f(\mathbf{p}+t \mathbf{v})$, we write the argument as $x(t)= \mathbf{p}+t \mathbf{v}$ and then apply the chain-rule in the definition?
PS: please keep the answer at the level of a physics undergraduate.
Your intuition is correct: in order to find the directional derivative at $\mathbf{p}$ in the direction $\mathbf{v}$, consider a smooth curve $t\mapsto x(t)$ that at $t=0$ passes through $\mathbf{p}$ with velocity $\mathbf{v}$. Now compute the rate of change of $f$ along this curve at time $t=0$, i.e. compute $F'(0)$ where $F(t)=f(x(t))$. Use the chain rule and you are done.