Is there a cubic polynomial $f(x)$ with real coefficients such that $f$ is monotonic and $f(x)=f^{-1}(x)$ has more than $3$ real roots?

Question

Does there exist a cubic polynomial $f(x)$ with real coefficients such that $f$ is monotonic (when regarded as a function from $\mathbb{R}$ to $\mathbb{R}$), and such that the equation $f(x)=f^{-1}(x)$ has more than $3$ real roots?

I couldn't find such an $f$, but I couldn't prove that no such $f$ exists.

It's clear that any such $f$ must be monotonically decreasing.

Answer 1

Yep! Consider $f(x)=-(x+1)^3$. There are five real roots to $f(x)=f^{-1}(x)$.

Answer 2

EDIT: this solution does not quite work.

Apply $f$ on both side we get $f^{2}(x)=x$.

If $f$ is increasing, for any root $x$ then $x<=f(x)<=f^{2}(x)$ or $x>=f(x)>=f^{2}(x)$ so either way the equality hold throughout and we get $f(x)=x$, in other word $f(x)-x=0$. Since $f$ is a cubic polynomial so is $f(x)-x$, and hence it can't have more than 3 roots.

EDIT: fix the problem with decreasing function. Though this proof merely prove that there is not more than $3$ pairs of values mapping into each other, not sure that's enough.

EDIT: actually not fixed, the case $t=0$ is not a double root.

If $f$ is strictly decreasing, then write $y=f(x)$. Using the change of variable $z=\frac{x+y}{2},t=\frac{x-y}{2}$ we have $x=z+t,y=z-t$ so the graph $f(x)-y=0$ become $f(z+t)-z+t=0$ which is a cubic polynomial in $z,t$, which we consider to be a cubic polynomial in $t$ with coefficients being at most cubic polynomial in $z$. At the root of $x=f(f(x))$ we have $y=f(x),x=f(y)$, so at these values we have a fixed $z$ such that the above equation has 2 real roots $t$, one is the negation of the other; if we also assume $x!=y$ then $t!=0$ and these are 2 distinct roots. Rescale by a constant if needed so that the leading coefficient is $1$ we get that the polynomial is of the form $t^{3}-P(z)t^{2}+Q(z)t-R(z)=0$ where $P(z)$ has degree at most $1$, $Q(z)$ has degree at most $2$ and $R(z)$ has exactly degree at most $3$ (in fact $3$ since the leading coefficient must be $1$ also). If $z$ is such that this polynomial has $2$ real roots that sum up to $0$, let them be $a,-a,b$ then $a+(-a)+b=P(z)$ so $b=P(z)$, $a(-a)+(-a)b+ba=Q(z)$ so $-a^{2}=Q(z)$ and $a(-a)b=R(z)$ so $Q(z)P(z)=R(z)$. Hence for $z$ be such that $2$ such roots for $t$ exist then $z$ must satisfy the polynomial $P(z)Q(z)-R(z)=0$. But $P(z)Q(z)-R(z)$ has at most degree $3$ so it either has only at most $3$ roots or it is identically $0$. It can't be exactly $0$ otherwise there are always exactly $2$ roots for every $z$, contradicting the fact that $f$ is cubic. Hence there are at most $3$ values of $z$ that work.