I'm not sure whether the above is true as it came up while I was trying to answer another question. I have broken the attempted proof into two cases, but am stuck on the second.
Suppose $f$ is reducible over $\mathbb{F}_{p^2}$. This may occur in two ways: 1) $f$ has a zero $\alpha \in \mathbb{F}_{p^2}$ or 2) $f$ factors as the product of irreducible quadratic polynomials.
Case 1: Suppose $\alpha \in \mathbb{F}_{p^2}$ is a zero of $f.$ As $f$ is monic and irreducible over $\mathbb{F}_p$, it holds that $f$ is the minimum polynomial of $\alpha$ over $\mathbb{F}_p$. This implies that $\mathbb{F}_p(\alpha) \cong \mathbb{F}_p[x]/(f)$. In particular, $[\mathbb{F}_p(\alpha):\mathbb{F}_p]=4$. The field $\mathbb{F}_{p^2}$ contains both $\mathbb{F}_p$ and $\alpha$, therefore $\mathbb{F}_{p}(\alpha) \subset \mathbb{F}_{p^2}.$ Hence \begin{equation*} 4= [\mathbb{F}_p(\alpha):\mathbb{F}_p] \leq [\mathbb{F}_{p^2}:\mathbb{F}_{p}]=2, \end{equation*} a contradiction.
Case 2: Suppose $f=gh \in \mathbb{F}_{p^2}[x],$ with $\deg(g)=\deg(h)=2.$ We may assume $g$ and $h$ to be irreducible, as if one is not then we are back in the first case. I don't see how to proceed from here, so I'm looking for a hint or counter-example.
In fact $f$ cannot be irreducible over $\mathbb F_{p^2}$. If $\alpha$ is a root of $f$, then $\mathbb F_p(\alpha)$ is a degree 4 extension of $\mathbb F_p$, and so is a degree $2$ extension of $\mathbb F_{p^2}$, implying that the minimal polynomial of $\alpha$ over $\mathbb F_{p^2}$ has degree 2. This polynomial will divide $f$.
(In Daniel Schleper's comment above, you can see explicitly what this minimal polynomial is.)