A.1 First we show $f(x)$ is reducible mod $p$ for all $p$.
Taking $y=x^2$ ; $f(x)$ becomes $y^2+(4a+2)y+1$. Solving the equation $y^2+(4a+2)y+1=0$ we get $$ y=-(2a+1)\pm2\sqrt{a(a+1)} $$ Now replacing $y$ by $x^2$ we have $$ x=\pm \sqrt{-(2a+1)+2\sqrt{a(a+1)}} = \pm \ (i\sqrt{a} + i\sqrt{a+1}) $$ & $$ x=\pm \sqrt{-(2a+1)-2\sqrt{a(a+1)}} = \pm \ (i\sqrt{a} - i\sqrt{a+1}) $$
Hence we have obtained all the roots of the polynomial $f(x)$ say $x_1, x_2, x_3, x_4$ where
$$ x_1= (i\sqrt{a} + i\sqrt{a+1}) \ ; \ x_3= -(i\sqrt{a} + i\sqrt{a+1}) $$ $$ x_2= (i\sqrt{a} - i\sqrt{a+1}) \ ; \ x_4= -(i\sqrt{a} - i\sqrt{a+1}) $$
A.2 The three ways of expanding this quartic are as follows:
$\{(x-x_1)(x-x_2)\}\{(x-x_3)(x-x_4)\}=(x^2+1-2\sqrt{a}ix)(x^2+1+2\sqrt{a}ix)\\ = (x^2+1)^2-(2\sqrt{a}ix)^2 \\ = (x^2+1)^2-2^2\mathbf{(-a)}x^2$
$\{(x-x_1)(x-x_4)\}\{(x-x_2)(x-x_3)\}=(x^2-1-2\sqrt{a+1} \ ix)(x^2-1+2\sqrt{a+1} \ ix)\\ = (x^2-1)^2-(2\sqrt{a+1} \ ix)^2\\ =(x^2-1)^2-2^2 \ \mathbf{(-(a+1))} \ x^2$
$\{(x-x_1)(x-x_3)\}\{(x-x_2)(x-x_4)\}\\ =(x^2+(2a+1)+2\sqrt{a(a+1)})(x^2+(2a+1)-2\sqrt{a(a+1)})\\ =(x^2+(2a+1))^2-(2\sqrt{a(a+1)})^2\\ =(x^2+(2a+1))^2-2^2\mathbf{(a(a+1))}$
A.3 Now the final piece.
$f(x)$ can be factored in any of the above 3 ways. Consider any prime $p$.
If $\mathbf{-a}$ is a square element in $\mathbb{F}_p$; there exist $b \in \mathbb{F}_p$ such that $-a=b^2$. So factoring $f(x)$ as in form A.2.1. Hence considering $f(x)$ in $\mathbb{F}_p[x]$ we have $$ f(x)=(x^2+1)^2-2^2(-a)x^2=(x^2+1)^2-(2bx)^2=(x^2+1-2bx)(x^2+1+2bx) $$ Hence $f(x)$ is reducible modulo $p$ if $-a$ is a square element.
If $\mathbf{-(a+1)}$ is square element in $\mathbb{F}_p$, there exist $c \in \mathbb{F}_p$ such that $-(a+1)=c^2$. So we factor $f(x)$ as in form A.2.2. Now considering $f(x)$ in $\mathbb{F}_p[x]$ we have $$ f(x) =(x^2-1)^2-2^2 \ \mathbf{(-(a+1))} \ x^2=(x^2-1)^2-(2cx)^2=(x^2-1+2cx)(x^2-1+2cx) $$ Hence we get $f(x)$ is reducible modulo $p$ if $2$ is a square element.
Now if both $\mathbf{-a}$ and $\mathbf{-(a+1)}$ are non square elements in $\mathbb{F}_p$ then $\mathbf{a(a+1)}$ is a square element in $\mathbb{F}_p$. Hence factoring $f(x)$ as in the form A.2.3 we get that $f(x)$ is reducible modulo $p$.
The following gives a mild generalization, but the idea is essentially what has been discussed.
Proposition 1. Let $f(x)=x^4+ax^2+b,~a,b\in {\mathbb N},$ where $a^2-4b$ is a non-square, and either (1) $b$ is a non-square, or (2) $b$ is a square and $\pm2\sqrt{b}-a$ is a non-square. Then $f(x)$ is irreducible over ${\mathbb Z}.$
Proof. Assume that $f$ satisfies the conditions. Clearly $f$ has no rational roots (in fact, no real roots). By comparison of coefficients, if $f$ were reducible, it must be of the form $$f(x)=(x^2+cx+d)(x^2-cx+e), c,d,e\in {\mathbb Z},$$ where $c\neq 0$ implies $e=d$. Necessarily $c\neq 0$ and $e=d$, since $a^2-4b$ is a non-square. Now by comparison of coefficients again, one has $$2d-c^2=a,d^2=b.$$ If $b$ is not a square, one immediately gets a contradiction. If $b=d^2$, then $d=\pm \sqrt{b},$ and one would have $\pm 2\sqrt{b}-a=c^2,$ contradicting the assumption that $\pm2\sqrt{b}-a$ is not a square. It follows that $f$ is irreducible. $\Box$
Proposition 2. Let $f(x)=x^4+ax^2+1$ with $a\in {\mathbb N}$ such that $a>2$ and $a^2-4$ is a non-square. Then $f$ is irreducible over ${\mathbb Z}$, but $f$ is reducible over ${\mathbb F}_p$ for any prime $p$.
Proof.
Part 1. $f(x)$ is irreducible over ${\mathbb Z}$.
This follows from Proposition 1, since $a^2-4$ is not a square, and $\pm 2(1)-a<0$ is not a square.
Part 2. $f(x)$ is reducible over ${\mathbb F}_p$ for any prime $p$.
Case 0. The case $p=2$ is immediate, since $$x^4+1=(x^2+1)^2$$ and $$x^4+x^2+1=(x^2+x+1)^2.$$ Assume from now on that $p$ is odd. Using the Legendre symbol, it suffices to consider the following cases.
Case 1. $p\mid D$ or $\left(\frac{D}p\right)=1,$ where $D=a^2-4=(2-a)(-2-a).$ Writing $D=u^2$ over ${\mathbb F}_p,$ one has $$x^4+ax^2+1=\left(x^2-\frac{-a+u}2\right)\left(x^2-\frac{-a-u}2\right),$$ which might be factored further.
Case 2. Either $\left(\frac {2-a}p\right)=1$ and $\left(\frac {-2-a}p\right)=-1,$ or $\left(\frac {-2-a}p\right)=1$ and $\left(\frac {2-a}p\right)=-1.$
Over ${\mathbb F}_p,$ one seeks to factor $$x^4+ax^2+1=(x^2+cx+d)(x^2-cx+d),$$ which is equivalent to $$2d-c^2=a,d^2=1$$ $$\Leftrightarrow (d=1~{\rm and~}2-a=c^2)~{\rm or~}(d=-1~{\rm and~}-2-a=c^2).$$ This means that when $\left(\frac {2-a}p\right)=1,$ one can set $d=1$, and when $\left(\frac {-2-a}p\right)=1,$ one can set $d=-1.$ The $c$ can be solved accordingly. This completes the proof of the reducibility of Case 2, and Proposition 2. $\Box$
To answer the question when $f(x)=x^4+(4a+2)x^2+1,~a\in {\mathbb N},$ just apply Proposition 2. One has $$(4a+2)^2-4=16a(a+1),$$ which is not a square, since $\gcd(a,a+1)=1$ and $a,a+1$ cannot be simultaneously squares. The other condition is obviously satisfied. Hence $f(x)$ is irreducible over $\mathbb Z$, and reducible over ${\mathbb F}_p$ for any prime $p.$
Remark. It may be tempting to assert the reducibility of $f(x)$ over ${\mathbb F}_p$ for any prime $p$ under the conditions in Proposition 1. But this is not true. For example, let $f(x)=x^4+5x^2+2.$ Then $f(x)$ is irreducible both over ${\mathbb Z}$ and ${\mathbb F}_5.$