We consider the function f(x) = arctan(x), we know that the newton iteration might diverge for some starting value too far from the zero. But it might also lead to an oscillation between two values as you can see in the following graph :
the goal of the problem is to find an expression for the function g. I've tried a lot of algebra but didn't manage to find any answer.
The iteration in Newton's method for $\arctan x$ is
$$ x_{n+1} = x_n - (1+x_n^2) \arctan x_n. $$
The red dots in your graph, plus the zero of $\arctan x$ at $x=0$, are the points $x_0$ for which $x_2 = x_0$; they cause the sequence $(x_n)$ to oscillate back and forth between two values. Because $\arctan x$ is odd, it is also true for these points $x_0$ that $x_1 = -x_0$.
Plugging this into the iteration we find that the red dots (plus the zero at $x=0$) satisfy
$$ -x_0 = x_0 - (1+x_0^2) \arctan x_0, $$
or, rearranging,
$$ 2x_0 - (1+x_0^2) \arctan x_0 = 0. $$
In other words, the red dots, plus the zero of $\arctan x$ at $x=0$, are the zeros of the function
$$ g(x) = 2x - (1+x^2) \arctan x, $$
which is the red curve in your plot.