Irreducible linear set of quadratics over $\Bbb F_p$

Question

Given $a,b\in\Bbb F_p$, denote $$S(a,b)=\big\{(a+\beta)x^2+(b-\beta)x+1\in\Bbb F_p[x]:\beta\in\Bbb F_p\big\}.$$

Denote $$S(a,b)_\mathrm{red}=\big\{g(x)\in S(a,b):g(x)\text{ is reducible}\big\}.$$

What is a good upper and lower bound for $$E_{\max}=\max_{\substack{a,b\in\Bbb F_p }}\big|S(a,b)_\mathrm{red}\big|?$$

What is a good upper and lower bound for $$E_{\min}=\min_{\substack{a,b\in\Bbb F_p }}\big|S(a,b)_\mathrm{red}\big|?$$

What is a good upper and lower bound for $$E_\mathrm{avg}=\frac{\displaystyle\sum_{\substack{a,b\in\Bbb F_p}} \big| S(a,b)_\mathrm{red} \big|}{p}?$$

Is there sharp estimates?

Can $E_\mathrm{avg}$ or $E_{\max}$ be $O(\log p)$ or both just $\Theta(\frac{p}2)$?

Can $E_{\min}=0$ be possible? If so with what probability?

Answer 1

Here's how it goes. Recalling some observations from the comments, and then adding more steps:

1. The set $S(a,b)$ only depends on the value of $a+b$, because $$S(a,b)=\{f(x)=a_2x^2+a_1x+1\in\Bbb{F}_p[x]\mid f(1)=1+a+b\}.$$
2. Ignoring the case $a_2=0$ (i.e. when $f(x)$ is linear), the polynomial $f(x)$ is irreducible if and only if when its reciprocal polynomial $$ \tilde{f}(x):=x^2f(\frac1x)=x^2+a_1x+a_2 $$ is irreducible.
3. The polynomial $\tilde{f}(x)$ is irreducible, if and only if the polynomial $$g(x):=f(x+1)=x^2+(a_1+2)x+(a_1+a_2+1)$$ is irreducible.
4. Therefore there is a bijection from the set of irreducible quadratic polynomials in $S(a,b)$ to the set of monic irreducible quadratic polynomials $$g(x)=x^2+Ax+B$$ with $B=a+b+1$.
5. Altogether there are exactly $(p^2-p)/2$ monic quadratic irreducible polynomials in $\Bbb{F}_p[x]$. These are exactly the minimal polynomials of elements of the set $\Bbb{F}_{p^2}\setminus\Bbb{F}_p$. There are $p^2-p$ such elements, but the conjugate pairs, $\alpha$ and $\alpha^p$, share the same minimal polynomial.
6. The constant term of such a minimal polynomial $$m_\alpha(x)=(x-\alpha)(x-\alpha^p)=x^2-[\alpha+\alpha^p]x+\alpha^{p+1}$$ is known as the norm of $\alpha$, $N(\alpha)=\alpha^{p+1}\in\Bbb{F}_p$.
7. The norm map $N:\Bbb{F}_{p^2}^*\to\Bbb{F}_p^*$ is a surjective homomorphism of cyclic groups. A calculation of orders of theses groups immediately tells us that $\operatorname{Ker}N$ is a subgroup of size $p+1$. Therefore any non-zero $B\in\Bbb{F}_p$ occurs as the norm of exactly $p+1$ elements of $\Bbb{F}_{p^2}$.
8. For all $c\in\Bbb{F}_p$ we have $N(c)=c^2$. So such a $B\in\Bbb{F}_p^*$ occurs as the norm of an element of $\Bbb{F}_p$ if and only if it is a square in $\Bbb{F}_p^*$. When that is the case we have $B=N(c)$ for exactly two elements $c\in\Bbb{F}_p$, namely $c=\pm\sqrt{B}$.
9. The number of irreducible quadratics $g(x)$ from step 4 is thus equal to $(p+1)/2$ when $B=a+b+1$ is a non-square, and $(p-1)/2$ when $B$ is a non-zero square in $\Bbb{F}_p$.
10. Therefore the number of irreducible quadratic polynomials in the set $S(a,b)$ is $$ |S(a,b)|_{irred}=\begin{cases} 0&,\ \text{if $a+b+1=0$,}\\ \dfrac{p+1}2&,\ \text{if $a+b+1$ is a non-zero square of $\Bbb{F}_p$},\\ \dfrac{p-1}2&,\ \text{if $a+b+1$ is a non-square of $\Bbb{F}_p$}. \end{cases}. $$

I'm sure you can do the rest.