I was reading the definition of naturality from AT (on pg. 127), it says:
For example, to say that the long exact sequence of a pair is natural means that for a map $f:(X, A) \rightarrow(Y, B),$ the diagram
\begin{array}{c} \cdots \longrightarrow H_{n}(A) \stackrel{i_{*}}{\longrightarrow} H_{n}(X) \stackrel{j *}{\longrightarrow} H_{n}(X, A) \stackrel{\partial}{\longrightarrow} H_{n-1}(A) \longrightarrow \cdots \\ \downarrow f_{*}\quad \quad \downarrow f_{*}\quad \quad \downarrow f_{*} \quad \quad \quad \quad \downarrow f_{*} \\ \cdots \longrightarrow H_{n}(B) \stackrel{i *}{\longrightarrow} H_{n}(Y) \stackrel{j *}{\longrightarrow} H_{n}(Y, B) \stackrel{\partial}{\longrightarrow} H_{n-1}(B) \longrightarrow \cdots \end{array}
is commutative. Commutativity of the squares involving $i_{*}$ and $j_{*}$ follows from the obvious commutativity of the corresponding squares of chain groups, with $C_{n}$ in place of $H_{n}$. For the other square, when we defined induced homomorphisms we saw that $f_{\not} \partial=\partial f_{f}$ at the chain level. Then for a class $[\alpha] \in H_{n}(X, A)$ represented by a relative cycle $\alpha$, we have $f_{*} \partial[\alpha]=f_{*}[\partial \alpha]=\left[f_{\text {ł }} \partial \alpha\right]=\left[\partial f_{\neq} \alpha\right]=\partial\left[f_{\neq} \alpha\right]=\partial f_{*}[\alpha]$.
But then on pg.210, Proposition 3.10., proved the naturality of cup product as can be seen below:
Proposition 3.10. For a map $f: X \rightarrow Y$, the induced maps $f^{*}: H^{n}(Y ; R) \rightarrow H^{n}(X ; R)$ satisfy $f^{*}(\alpha \smile \beta)=f^{*}(\alpha) \smile f^{*}(\beta),$ and similarly in the relative case.
Proof: This comes from the cochain formula $f^{\#}(\varphi) \smile f^{\#}(\psi)=f^{\#}(\varphi \smile \psi)$ : $$ \begin{aligned} \left(f^{\#} \varphi \smile f^{\#} \psi\right)(\sigma) &=f^{\#} \varphi\left(\sigma \mid\left[v_{0}, \cdots, v_{k}\right]\right) f^{\#} \psi\left(\sigma \mid\left[v_{k}, \cdots, v_{k+\ell}\right]\right) \\ &=\varphi\left(f \sigma \mid\left[v_{0}, \cdots, v_{k}\right]\right) \psi\left(f \sigma \mid\left[v_{k}, \cdots, v_{k+\ell}\right]\right) \\ &=(\varphi \smile \psi)(f \sigma)=f^{\#}(\varphi \smile \psi)(\sigma) \end{aligned} $$
Could someone tell me how this proposition is a proof of naturality of cup product according to the definition of naturality given above?
The quote from page 127 of Hatcher is not a definition of naturality, just an example.
Naturality is a general concept from category theory (which amusingly, at time of writing, is the one tag that this question doesn't use among all the other less relevant tags). Recall that a natural transformation $\eta: F \Rightarrow G$ between functors $F, G: \mathcal{C} \to \mathcal{D}$ is a collection of morphisms $\{\eta_X: FX \to GX\}$ in $\mathcal{D}$ indexed by the objects $X$ in $\mathcal{C}$. The key compatibility condition ("naturality") that these morphisms have to satisfy is that for any morphism $f: X \to Y$ in $\mathcal{C}$, the following diagram commutes: $$\require{AMScd} \begin{CD} FX @>{Ff}>> FY \\ @V{\eta_X}VV @VV{\eta_Y}V \\ GX @>>{Gf}> GY. \end{CD}$$
For example, the connecting homomorphism $\partial$ in the long exact sequence in homology is a natural transformation between the functors $(X,A) \mapsto H_n(X,A)$ and $(X,A) \mapsto H_{n-1}(A)$ from the category of pairs of spaces to the abelian groups. The naturality condition is that $\partial \circ f_* = f_* \circ \partial$, i.e., the square you drew involving $\partial$ commutes.
For another example, the cup product $\smile$ is natural transformation between the functors $X \mapsto H^p(X) \otimes H^q(X)$ and $X \mapsto H^{p+q}(X)$ from the category of spaces to abelian groups. The naturality condition is $\smile \circ (f^*, f^*) = f^* \circ \smile$, i.e., $f^* \alpha \smile f^* \beta = f^*(\alpha \smile \beta)$ for any two cohomology classes $\alpha, \beta$.