Problem: Suppose $F:N\rightarrow M$ is a smooth map between manifolds. Let $[f]_{F(p)}\in C_{F(p)}^{\infty}(M)$ . Then, $[f\circ F]_p\in C^{\infty}_p(N)$.
Note that $C_p(N)$ is the (real) vector space of germs at $p$.
I am wondering why the composition $f\circ F$ makes sense, may someone elaborate, please?
Note that $f$ is a map from an open subset $U$ (that contains p) of $M$ to $\mathbb{R}$.
Is what really meant by $f\circ F$, the map $f\circ g$, where $g:F^{-1}(U)\rightarrow U$ is given by $g(x)=F(x)$? If so, why not just write $f\circ g$? (**)
Therefore, my attempt considering $(**)$:
If $F$ in $f\circ F$, is the map $F|_{F^{-1}(U)}:F^{-1}(U)\rightarrow U$, then this is a smooth map. In which case, $[f\circ F]_p\in C_p^{\infty}(N)$, since the composition of smooth maps is smooth. ($F^{-1}(U)$ is open in $N$ by continuity of smooth maps and is therefore a smooth manifold)
Is everything I have written correct?
The composition $f\circ F$ does not make sense, but the germ $[f\circ F]_p$ does. Indeed, you are right to note that $f\circ F$ makes sense in a neighborhood of $p$, namely on $F^{-1}(U)$ (and that is all we need to make sense of its germ at $p$). To be more precise, we could write $f\circ F\vert_{F^{-1}(U)}$. The reason why we do not write it this way is simplicity. You want to write down a definition of the differential of $F\colon N\rightarrow M$ at a point $p\in N$. We can define this map as follows: $$dF\vert_p\colon T_pN\rightarrow T_{F(p)}M,\,X\mapsto([f]_{F(p)}\mapsto X([f\circ F]_p)).$$ Why not write $[f\circ F\vert_{F^{-1}(U)}]_p$? Because there isn't any $U$ in this expression! So if you wanted to do as such, you would at first have to specify that the representative $f$ is defined on a set $U$. You could, for example, write $X\mapsto([f\colon U\rightarrow\mathbb{R}]_{F(p)}\mapsto X([f\circ F\vert_{F^{-1}(U)}\colon F^{-1}(U)\rightarrow\mathbb{R}]_p))$ instead, but this term is very unwieldy. Even then, this doesn't mention that $U$ ought to be an open neighborhood of $F(p)$ in $M$. So, as per usual, this is a slight abuse of notation as to not completely clutter notation with details that should be implicitly understood (indeed, there really is no other way to interpret $f\circ F$ and the fact that you naturally arrived at the right interpretation is testament to this). In fact, most people would be even more casual and just write $X\mapsto(f\mapsto X(f\circ F))$ instead, making it implicit that $f$ is supposed a germ (but, what else should it be given we are defining a map on a space of germs?). So yeah, there is something underhanded going on here and it's good that you are tripping over it, but it's also the type of thing that you only need to sort out once and then you can cease worrying about it.
Lastly, there is another subtlety here that you need to consider, namely the well-definedness of this map. That is, if $f$ and $f^{\prime}$ define the same germ at $F(p)$, then $f\circ F$ and $f^{\prime}\circ F$ define the same germ at $p$ (convince yourself ot this).