I'm trying to solve the following problem :
"Let $f$ be a quartic polynomial with a positive leading coefficent, and all coefficients are real numbers. Let $m$ be the number of real solution of $f(x)=x$, and $n$ be the number of real solution of $f(f(x))=x$. If $m,n$ are natural numbers, then how many ordered pairs of $(m,n)$ are possible?"
When $m=1$, it follows that $f(x) \ge x$, which in turn implies $f(f(x)) \ge f(x) \ge x $ and equality holds iff $f(x)=x$, thus $n=1$.
But in other cases, when $m=2,3,4$ I can't handle it. The only guess I have is as follows. If $f(x)=x$, then $f(f(x))=x$, thus $n \ge m$. And if $f(f(x))=x$ and $f(x) \ne x$, then there exists distinct $a$ and $b$ such that $f(a)=b , f(b)=a$. Thus $n$ should be $m+(even number)$.
Is there any other hints can I get? I desperately need help now. I'm in trouble..
Consider a simplified problem.
$f(x)=ax^2+bx+c$
$f(x)=x\implies ax^2+(b-1)x+c=0$
$\implies x= \frac{(1-b)\pm \sqrt{(b-1)^2-4ac}}{2a}$
$\frac{(b-1)^2}{4a}\ge c$
By FTA, we know there are at most 2 real solutions. This occurs iff $(b-1)^2-4ac>0$. There might be only one solution if $(b-1)^2=4ac$, and no real solutions if $(b-1)^2-4ac<0$. The term under the radical is the discriminant.
Suppose $b=0$ and $a=1$. Then the discriminant is $\sqrt{1-4c}$ So there are no fixed points for $c>1/4$, one distinct solution for $c=1/4$ and two solutions for $c<1/4$.
$f(f(x))=(x^2+c)^2+c=x^4+2cx^2+c^2+c=x$. Since $f(x)=x$ gives solutions of this equation too,
$x^4+2cx^2-x+c^2+c=(Ax^2+Bx+C)(x^2-x+c)$
$A=1, C=c+1, B=A=1$
So $x=\frac{-1\pm \sqrt{-3-4c}}{2}$ gives potentially $2$ new solutions. $-3-4c\ge 0 \implies -3/4 \ge c$
If $f(x)=x$, solutions are called fixed points. If $f(f(x))=x$ and $f(x) \ne x$, solutions are called periodic points of prime period 2.
In this simplified case, there can be $2,1$ or $0$ distinct fixed points and $2,1$ or $0$ period 2 points, (ignoring solutions of $f(f(x))=x$ where $f(x)=x$ ).
If $f(x)$ is a 4th degree polynomial, the discriminant is more complicated: 4th degree polynomial discriminant
In the second order version the solution you seek is a subset of the Cartesian Product $\{2,1,0\} \times \{4, 3,2 ,1, 0\}$
$(2,0)$ is not a possible ordered pair since since 2 solutions for $f(x)=x$ implies at least two solutions for $f(f(x))=x$. Note iff there is only 1 solution of $f(x)=x$, where $f(x)=x^2+c$, there is only one solution for $f(f(x))=x$, i.e. the one distinct fixed point, and since the discriminant of $f(f(x))<0$ in that case, no solutions having period 2.
Also keep in mind you can find possible solutions graphically.
Fixed points are intersections of $y=x$ and $y=f(x)$.
Period two points are $y=x$ intersecting $y=f(f(x))$.
Hope this helps. For more look up algorithms for finding fixed points and periodic points of prime period 2.