$f(f(x))=f(x),$ for all $x\in\Bbb R$ suppose $f$ is differentiable, show $f$ is constant or $f(x)=x$

Question

$f(f(x))=f(x),$ for all $x\in\Bbb R$ suppose $f$ is differentiable, show $f$ is constant or $f(x)=x$

Clearly, $f'(f(x))f'(x)=f'(x)$. This implies for each $x$, $f'(f(x))=1$, or $f'(x)=0$. But this is not enough.

Answer 1

Since $f(f(x))=f(x)$, $y=f(x)$ is a valid input to the function, implying that the function's range is a subset of its domain. With that in mind, $f'(f(x))$ is analogous to $f'(y)$ for some $y$ in the range of $f$.

$f'(y)=1$ or $f'(x)=0$. In the first case, this immediately shows that $f(y)=y+C$ for all $y$ in each simply connected subset of the range of $f$, but $C=0$ for all these regions since $f(y)=y$ by the first equation. Now since $f$ is differentiable everywhere in $\mathbb{R}$, the only continuation of the derivative for some $f'(x),x\notin\operatorname{Range}(f)$, that preserves the continuity of $f$ on the border of every connected region of its range, is $1$, showing that $f(x)=x$ for all $x\in\mathbb{R}$.

In the second case, it immediately shows that $f(x)=k$, for some constant $k$, which applies to the whole domain of $f$ - $\forall x\in\mathbb{R}$.

Answer 2

I'm not able to follow the argument given in the existing answer. It may be perfectly valid, but I don't understand what's being said at several points. So let me give my own answer.

Suppose $f : \mathbb{R} \to \mathbb{R}$ is a differentiable function such that $f(f(x)) = f(x).$ Differentiating, we have $$f'(f(x))f'(x) = f'(x)$$ which we can rewrite as $$f'(x)[f'(f(x)) - 1] = 0$$

So for each $x \in \mathbb{R}$ we have either $f'(x) = 0$ or $f'(f(x)) = 1$.

Since $f$ is differentiable, it is continuous. Since $\mathbb{R}$ is nonempty and connected, so is its continuous image $Y := f(\mathbb{R})$.

Let $y \in Y$, say $y = f(x)$. We must have either $f'(y) = 0$ or $f'(f(y)) = 1$. But since $y = f(x)$, $$f(y) = f(f(x)) = f(x) = y.$$ So in all we have either $f'(y) = 0$ or $f'(y) = 1$ for each $y \in Y$.

Derivatives of differentiable functions need not be continuous, but they at least satisfy the intermediate value property. If $f'(y_1) = 0$ and $f'(y_2) = 1$ for some points $y_1, y_2 \in Y$, it would necessarily be the case that there was some point $y_3$ in between them with $f'(y_3) = 1/2$. Since $Y$ is connected, though, this point $y_3$ would necessarily be in $Y$ as well, contradicting the fact that $f'$ only takes on the values 0 and 1 on $Y$.

It follows that either $f'(y) = 0$ for all $y \in Y$ for $f'(y) = 1$ for all $y \in Y$.

In the former case, we have $f'(f(x)) = 0$ for all $x$, so in particular $f'(f(x)) \neq 1$ for all $x$, so we must have $f'(x) = 0$ for all $x$. But this means that $f$ is constant.

In the latter case, we have $f'(y) = 1$ for all $y \in Y$. First, since $f'$ is nonzero at a point, $f$ is nonconstant, so $Y$ must be an interval of positive length. Second, since $f'(y) = 1$ on this interval, there is some constant $C$ for which $f(y) = y + C$ for all $y \in Y$. That is, for all $x$, $f(f(x)) = f(x) + C$. But $f(f(x)) = f(x)$, so this says that $f(x) = f(x) + C$, i.e. $C = 0$. In other words, $f(y) = y$ for all $y \in Y$.

I want to claim in this case that $Y$ is necessarily all of $\mathbb{R}$. Suppose not. Write $a = \inf Y$ and $b = \sup Y$. Since $Y$ is an interval of positive length, we have in particular $a < b$. By assumption, one or both of these must be finite. Assume without loss of generality that $a$ is not $-\infty$.

Since $f(y) = y$ at least on the interval $(a, b)$ and $f$ is continuous, we must have $f(a) = a$. For $x < a$ we have $f(x) \geq a$ since $a$ is the infimum of the range of $f$. For $a \leq x < b$ we have $f(x) = x$.

But $f$ is supposed to be differentiable, so let's consider its derivative,

$$\lim_{h \to 0} \frac{f(x + h) - f(x)}{h}$$

The limit from the right is $$\lim_{h \to 0^+} \frac{f(x + h) - f(x)}{h} = 1$$ On the other hand, if we consider the limit from the left, $$\lim_{h \to 0^-} \frac{f(x + h) - f(x)}{h}$$ the numerator is nonnegative (since $f(x+h) \geq a$) while the denominator is a negative number. So the limit, should it exist, could at most be zero, and is definitely not equal to one.

This is a contradiction, so it must be the case that $a = -\infty$. Similarly, $b = +\infty$. So the $f(y) = y$ for all $y \in Y$, but $Y$ is all of $\mathbb{R}$ so $f$ is just the identity function.