The zero section is the map $\zeta: M \to E$ defined by $\zeta(p)=0_p \in E_p$.
Since $E \to M$ is a vector bundle, then for each $p \in M$ there exists a neighborhood $U$ of $p$ such that $\Phi: \pi^{-1}(U) \to U \times \mathbb R^k$ is a homeomorphism.
I want to show that $\zeta$ restricted to $U$ is continuous so that for each point in $M$ there exists a neighborhood such that the restriction of $\zeta$ to this neighborhood is continuous. This tells us $\zeta$ is continuous.
Since, $\Phi \circ \zeta (p)=\Phi(0_p)=(p,0)$, then $\zeta((p,0))=\Phi^{-1}((p,0))$. This tells me $\zeta|_U=\Phi^{-1}|_{U \times \{0\}}$. But $U \times \{0\}$ is not open in $U \times \mathbb R^k$. So, we can't conclude $\zeta|_U$ is continuous.
How can I proceed in showing that $\zeta$ is continuous?
Observe that $(\Phi \circ \xi)(q) = \Phi(0_q) = (q,0)$ for all $q \in U$ because $\Phi$ is linear. Therefore, $\Phi \circ \xi$ is continuous. Since $\Phi$ is a homeomorphism, $\xi$ is continuous.