Please see this question for Lemma 58.4.
I don't really understand how Munkres concludes so quickly (in the last paragraph) that $F\circ G$ is a path homotopy between $f_0\ast c$ and $c\ast f_1$ (and also that $H\circ (F\circ G)$ is a path homotopy between the two paths he mentions). I can certainly write everything out and see that it's indeed true, but I have a feeling that there is some obvious reason for it. For example, I don't know how he uses $F\circ \beta_0=f_0, F\circ \beta_1=f_1,F\circ \gamma_0=F\circ \gamma_1=c$. Do these equalities make the facts I mentioned clear somehow (they may come up in the direct proof, but maybe it's immediately clear from them that the compositions give the homotopies between the mentioned paths)?
As stated in the proof $G:I\times I \rightarrow I\times I$ is a homotopy from $\beta_0 * \gamma_1$ to $\gamma_0 * \beta_1$. Now $F\circ G:I\times I \rightarrow X\times I$ is a homotopy of something surely. We want to show it is the path homotopy from $f_0*c$ to $c* f_1$. So we check the following using the equalities that you mentioned: $$F(G(s,0))=F(\beta_0*\gamma_1)=f_0*c$$ $$F(G(s,1))=F(\gamma_0*\beta_1)=c*f_1$$ $$F(G(0,t))=F(\beta_0(0))=F(0,0)=(f(0),0)=(f_0*c)(0)$$ $$F(G(1,t))=F(\beta_1(1))=F(1,1)=(f(1),1)=(c*f_1)(1). $$ Thus $F\circ G$ really is the homotopy we are looking for (from $f_0*c$ to $c* f_1$). To show $H\circ(F\circ G)$ is the desired homotopy we make the recall $f_0(s)=(f(s),0)$ thus $H\circ f_0=H(f(s),0)=h\circ f$ by the definition of $H$. Likewise $f_1(s)=(f(s),1)$ hence $H\circ f_1=H(f(s),1)=k\circ f$ by the definition of $H$ again. Thus we realize the equalities of Munkres' $$(H\circ f_0)*(H\circ c)=(h\circ f)*\alpha $$ $$(H\circ c)*(H\circ f_1)=\alpha *(k\circ f) .$$For the appropriate end points of the above paths we have $$((h\circ f)*\alpha)(0)=(h\circ f)(0)=h(x_0)=y_0=\alpha(0)=(\alpha*(k\circ f))(0) $$ $$((h\circ f)*\alpha)(1)=\alpha(1)=y_1=(k\circ f)(1)=(\alpha*(k\circ f))(1) $$Now to tie everything together, $H\circ(F\circ G):I\times I \rightarrow Y$ has the properties: $$H(F(G(s,0)))=H(f_0*c)=(H\circ f_0)*(H\circ c)=(h\circ f)*\alpha $$ $$H(F(G(s,1)))=H(c*f_1)=(H\circ c)*(H\circ f_1)=\alpha *(k\circ f)$$ $$H(F(G(0,t)))=H(f(0),0)=(h\circ f)(0)=h(x_0)=y_0$$ $$H(F(G(1,t)))=H(f(1),1)=(k\circ f)(1)=k(x_0)=y_1$$ and $H\circ (F\circ G)$ is the desired homotopy.