Let $p:E\to B$ be a covering map. Suppose $B$ is regular. I need to show that $E$ is regular.
It seems to me that requiring $p$ to be a covering map is too demanding, as in my following proof I use only the fact that covering maps are open. My attempt:
To show regularity, one needs for every $e\in E$ and $U$ open neighborhood of $e$ to find another neighborhood $V$ such that $e\in V\subseteq \overline{V}\subseteq U$.
Let $e_0\in E$, and let $U$ be an open neighborhood of $e_0$. Let $b_0 = p(e_0)$. Since $p$ is a covering map, it is open, hence $p(U)$ is an open neighborhood of $b_0$. Since $B$ is regular, we know there exists an open neighborhood $V$ such that $b_0\in V\subseteq \overline{V}\subseteq p(U)$. We therefore get that $$ e_0\in p^{-1}(V)\subseteq p^{-1}(\overline{V})\subseteq U $$ Since $p$ is continuous, $p^{-1}(V)$ is open and $p^{-1}(\overline{V})$ is closed, hence $\overline{p^{-1}(V)}\subseteq p^{-1}(\overline{V})\subseteq U$, and we get that $$ e_0\in p^{-1}(V)\subseteq \overline{p^{-1}({V})}\subseteq U $$
The problem is that your $U$ probably does not satisfy $p^{-1}(p(U))=U$, and so while you know $p^{-1}(\overline{V})\subseteq p^{-1}(p(U))$, this does not give you $p^{-1}(\overline{V})\subseteq U$.
So you'll need to use the fact that $p$ is a covering map and not just an open map. One way you can do this is to first choose a neighborhood $W\subseteq p(U)$ of $p(e)$ which is evenly covered by $p$, let $U'$ be the part of $U$ which is contained in the sheet of $p^{-1}(W)$ containing $e$, and choose $V$ such that $\overline{V}\subseteq p(U')$. You can then use not $p^{-1}(V)$ but only $p^{-1}(V)\cap U'$.