So I want to obtain the following form: For $x, p \in R^n$
$$\nabla f(x+p) = \nabla f(x) + \int^1_0 \nabla^2 f(x+tp)p \,dt$$ for $f$ twice continuously differentiable and $t \in (0,1)$.
Taylor's theorem in integral form is:
$$f(x) = f(a) + \int^x_a \nabla f(t) (x-t)\,dt$$ from here: https://www.math.upenn.edu/~kazdan/361F15/Notes/Taylor-integral.pdf
My approach is:
$$f(x+p) = f(x) + \int^{x+p}_x \nabla f(t) (x+p-t)\,dt$$ Reformulated as: $$f(x+p) = f(x) + \int^1_0 \nabla f(x+tp)(1-t)p\,dt$$ $$= f(x) + \int^1_0 \nabla f(x+tp) p \,dt - \int^1_0 \nabla f(x+tp) t p \,dt$$
But I am not sure how to take the derivatives of the integrals with respect to x; am I at a dead end? Should I instead, just begin with $\nabla f(x)$ in place of $f(x)$ when I apply Taylor's theorem?
Okay, I figured out the problem. Taylor's theorem in integral form is actually $$f(x) = f(a) + \int^x_a \nabla f(t) \,dt$$
From there, we have $$f(x+p) = f(x) + \int_0^1 \nabla f(x+tp)p\,dt$$
And we can substitute $\nabla f(x)$ for f(x) into the above statement.
EDIT:
The above is incorrect. Using https://www.math.washington.edu/~folland/Math425/taylor2.pdf (page 3), (with the same notation regarding $\alpha$)
$$f(x+p) = f(x) + \sum_{|\alpha| = 1} \frac{p^{\alpha}}{\alpha!} \int_0^1 \partial^{\alpha} f(x+pt) \,dt $$
$$f(x+p) = f(x) + \sum_{j=1}^n p_j \int_0^1 \partial_j f(x+pt) \,dt = f(x) + p \cdot \int_0^1 \nabla f(x+pt)\,dt $$
Taking the gradient of the above: $$\frac{\partial}{\partial_i} f(x+p) = \frac{\partial}{\partial_i} f(x) + \frac{\partial}{\partial_i} p \cdot \int_0^1 \nabla f(x+pt) \, dt = \frac{\partial}{\partial_i} f(x) + p \cdot \int^1_0 \frac{\partial}{\partial_i} \nabla f(x+pt) \,dt $$
We have overall that
$$\nabla f(x+p) = \nabla f(x) + \int^1_0 \nabla^2 f(x+pt) p \, dt$$