In physics, I have been taught that a vector field is just assigning an arrow at each point of a manifold.
In here I read a vector field is a mapping $$ v:C^\infty(M) \rightarrow C^\infty(M)$$ So an example for a vector field would be the derivative or directional derivative. I understand the directional derivative operation takes in a smooth function and spits out another smooth function, so it makes sense with the above definition.
My question is whether the integral operator, which again takes in a smooth function and spits out another smooth function(I hope this is correct, I am not sure about the rigor of this hypothesis) is another example for a vector field in the same sense as a derivative is a vector field.
You seem to missing the crucial point which is that the mapping $v\colon C^\infty(M)\to C^\infty(M)$ (which ought to be really defined in $G_p^\infty(M)$: the space of germs of smooth functions at $p$)needs to satisfy the Leibniz rule: $v(fg) = f(p)v(g) + g(p) v(f)$. So an integral operator is out of question. But the idea is that a tangent vector $v\in T_pM$ is completely characterized by the values $v(f)$, where $f$ ranges over $G^\infty_p(M)$. In less "fancy" lingo, a vector is characterized by how it acts as directional derivatives.
For instance, in $\Bbb R^3$, taking a directional derivative of a function in the direction of the vector $e_1 = (1,0,0)$ is the same thing as applying $\partial/\partial x$. For this reason, the vector field that assigns to each $p\in \Bbb R^3$ the tangent vector $(1,0,0)$ in $T_p(\Bbb R^3) \cong \Bbb R^3$ is denoted by $\partial/\partial x$.
Pretty much any differential geometry book will develop these definitions accordingly. See for example Chapter 3 in Loring Tu's An Introduction to Manifolds.