Finding a polynomial that satisfies a certain composition of functions

Question

Find all polynomials P(X) such that P(P(X)) = X. Then find all polynomials P(X) such that P(P(P(X)))= X.

Pretty much stuck, I have no experience with polynomials and the only thing I think that can help me is derivatives and the chain rule.

Answer 1

If $P$ has degree $2$ or more, then $P(P(x))$ will have higher degree, so we may rule that out. If $P$ is constant, then clearly $P(P(x)) \ne x$. So we are relegated to the case that $P$ is linear, so that $P(x) = ax + b$. We then find that

$$x = P(ax + b) = a(ax + b) + b = a^2 x + ab + b$$

This gives us conditions on $a$ and $b$ that we can use to narrow it down even further.

---

For an approach using calculus as you mentioned, note that

$$1 = (P(P(x))' = P'(P(x)) P'(x)$$

In particular, we see that $P'$ must be constant, verifying that $P$ can only be linear.

Answer 2

If $P(X)$ has degree $n$, then $P(P(X))$ has degree $n^2$. Do you see how this helps you?