Let $M$ be a smooth manifold of dimension $n$, and let $T(M)$ be the tangent bundle of $M$. Let $F$ be a smooth vector field, which is to say a smooth section of the canonical map $T(M) \rightarrow M$. If I understand the manifold structure on $T(M)$ correctly, smoothness means that for any chart $(U,\phi)$ of $M$, we can use $\phi$ to identify $T_p(M)$ with $\mathbb{R}^n$ for all $p \in U$, and under these simultaneous identifications, $F$ becomes a map $\phi(U) \rightarrow \mathbb{R}^n$ which is smooth.
Let $p_0 \in M$ and $t_0 \in \mathbb{R}$. Wikipedia defines an integral curve for the vector field $F$, passing through $p_0$ at time $t_0$, to be an open neighborhood $J$ of $t_0$, together with a smooth morphism $\alpha: J \rightarrow M$, such that $\alpha(t_0) = p_0$ and
$$\alpha'(t) = F(\alpha(t))$$
for all $t \in J$. I am confused on what this equality is saying. First, I do not understand what $\alpha'$ means as a map from $J$ to the manifold $M$. Maybe a chart $(U,\phi)$ containing the image of $J$ must be chosen, and then we can talk about the derivative of the composition $\phi \circ \alpha: J \rightarrow \mathbb{R}^n$. Even so, that derivative is a priori a collection of linear maps $\mathbb{R} \rightarrow \mathbb{R}^n$, so for each $t \in J$, $\alpha'(t)$ can be thought of as a linear map $\mathbb{R} \rightarrow \mathbb{R}^n$. On the other hand, $F(\alpha(t))$ identifies as an element of $\mathbb{R}^n$. I don't see in what sense these things are supposed to be equal. Are we also using the identification $\textrm{Hom}_{\mathbb{R}}(\mathbb{R},V) = V$ for any vector space $V$?
Let $x\in M$. One way of defining the tangent plane $T_xM$ at $x$ is to consider equivalent family of curves: Let $\alpha, \beta $ be two smooth mapping from an interval $J_1, J_2$ to $M$ so that $\alpha(c_1)=\beta (c_2)=x$. Then we say $\alpha\sim \beta$ if under a local coordinates $\phi : U\to \mathbb R^n$ with $x\in U$, we have
$$ (\alpha \circ \phi)'(c_1) = (\beta \circ \phi)'(c_2) .$$
The tangent plane at $x$ is the equivalent class (under $\sim$) of all such curves. In particular, if $\alpha J\to M$ is a smooth mapping, then by definition $\alpha$ defines, for each $t\in J$, an element in $T_{\alpha(t)}M$. This element is then denoted $\alpha'(t) \in T_{\alpha(t)}M$. So you are right that under each local coordinates $\phi$, $\alpha'(t)$ is presented locally as $(\alpha \circ \phi)'(t)$.