$f:M\to N$ is smooth iff $\forall g\in C^\infty(N)$, $g\circ f\in C^\infty(M)$

Question

Let $M$ and $N$ be smooth manifolds. As the title says, I am trying to show that a function $f:M\to N$ is smooth iff for all $g\in C^\infty(N)$, $g\circ f\in C^\infty(M)$. This question has already been asked here and here, but both seem to have a crucial mistake. The forward direction follows from the fact that a composition of smooth maps is smooth. For the backwards direction, I would like to consider charts $\varphi:U\to\mathbb{R}^n$ and $\psi:V\to\mathbb{R}^m$ for open $U\subseteq M$ and $V\subseteq N$ and say that $\psi\circ f\circ\varphi^{-1}$ is smooth because $\psi\circ f$ is. But this reasoning does not work since $\psi\notin C^{\infty}(N)$. Rather, $\psi\in C^{\infty}(V)$, so our assumption cannot be used on $\psi\circ f$ (which really denotes $\psi\circ f\vert_{f^{-1}(V)}$). So how could one go about showing this, and is it even true to begin with?