Here is the definition of pullback maps in linear algebra from Mclnerney's First Steps in Differential Geometry:
Let $\Psi: V\to W$ be a linear transformation and let $\Psi^*:W^*\to V^*$ be given by $(\Psi^*(T))(\textbf {v})=T(\Psi(\textbf{v}))$ for all $T\in W^*$ and $\textbf{v}\in V$. We call $\Psi^*$ the pullback map induced by $\Psi$ and $\Psi^*(T)$ the pullback of $T$ induced by $\Psi$.
What confuses me is that why $\Psi^*$ is called a pullback map. What does $\Psi^*$ "pull ", and where does $\Psi^*$ pull it back to? I feel confused about the way we call $\Psi^*$, and I think I won't really grasp this definition without resolving this confusion.
Starting with a linear functional $T$ on $W$, applying $\Psi^\ast$ gives a linear functional on $V$. So $\Phi^\ast$ pulls the domain of $T$ back from $W$ to $V$, while $\Phi$ moves forward points from $V$ to $W$.