Let $f(x) = x^3 - 3x + 1$. Let $\alpha, \beta, \gamma \in \mathbb{R}$ be the roots of $f$ with $\alpha > \beta > \gamma$. Let $g(x) = x^2 - 2$. We see that $\beta = g(\alpha)$, $\gamma = g(\beta)$ and $\alpha = g(\gamma)$ by calculation using $f$.
Why does this cyclic property hold? What is the relation between $f$ and $g$? Could you tell me its background or generalization?
On
This answer is factually incorrect: see the comments below.
Let $a$ be any root of $f(x)$. Then $a^3 - 3a + 1 = 0 \Rightarrow a^2 - 3 = -1/a \Rightarrow a^2 - 2 = -1/a +1$. Thus $g(a) = -1/a + 1$.
Let us find $g(g(g(a)))$, which is $g(g(-1/a + 1)) = g\left(-\frac{1}{-1/a+1} + 1 \right) = g\left(\frac{-a}{-1+a} + \frac{-1+a}{-1+a} \right) $ $ = g\left(\frac{-1}{-1+a}\right) = \frac{-1}{-1/(-1+a)} +1 = -1+a+1=a$. From this process, the roots of $f(x)$ in terms of $a$ are $a$, $-\frac{1}{a} + 1$ and $\frac{-1}{1+a}$. This means that cycling any root three times through $g(x)$ gives the original root, and these are the only three roots, since a cubic polynomial has at most three real roots.
The tricky part is the condition that $\alpha > \beta > \gamma$, where $\alpha$ is the largest root. In other words, how do we know that the cycling order isn't actually $\alpha, \gamma, \beta$? Observe that $(\alpha, \beta, \gamma)$, $(\beta, \gamma, \alpha)$ and $(\gamma, \alpha, \beta)$ represent the same cyclic permutation, and the other permutation $(\alpha, \gamma, \beta)$ also makes up three permutations, and because there are only $3! = 6$ in total, these are the only permutations.
So if the other permutation $(\gamma, \beta, \alpha)$ is true, then $\beta > \alpha$ (any other inequality would also hold for $(\gamma, \alpha, \beta)$), or $-\frac{1}{\alpha} + 1 > \alpha$, which leads to $\alpha < 0$ (*). By Descartes' rule of signs, $f(-x) = -x^3 + 3x + 1$ has one sign change, hence there is only one negative root. However, since there are three real roots from the question, the other two roots are positive ($0$ is not a root). Therefore $\alpha$ is not the largest root: contradiction, and the cycling order is $\alpha, \beta, \gamma$. Hence proved.
(*) Rearrange to get $\frac{1}{a} < 1- a$ as the inequality sign flips. You will have to split it into positive and negative cases to prove this algebraically, but this can be shown by a sketch.
I haven't actually shown that the values that $g(x)$ cycles through are the roots of $f(x)$! Vieta's formulas can be used to do this: prove that $a + g(a) + g(g(a)) = 0$, and do the same for $\alpha \beta + \beta \gamma + \gamma \alpha$ and $\alpha \beta \gamma$.
For any cubic polynomial $f(x)$ with three distinct roots $\alpha$, $\beta$, $\gamma$ you can find a quadratic polynomial $g(x)$ such that $g(\alpha)=\beta$, $g(\beta)=\gamma$, $g(\gamma)=\alpha$; it's just the matter of assuming $g(x)=rx^2+sx+t$ and solving three linear equations on coefficients $r$, $s$, $t$. What may be interesting is a method to find such a polynomial when you don't know the roots.
Let $\alpha$ be one of three distinct roots of $f(x) = x^3+bx^2+cx+d$. Then we can factorize $(x-\alpha)$ from $f(x)$ to get $$ f(x) = (x-\alpha)\big(x^2 + (b + \alpha) x +(\alpha^2 + b\alpha + c)\big) $$ which means that the other two roots of $f(x)$ are the roots of the polynomial $$ h(x) = x^2 + (b + \alpha) x +(\alpha^2 + b\alpha + c)$$ If we are looking for a binomial $g(x) = rx^2+sx+t$ such that $g(\alpha)$ is another root of $f(x)$, i.e. $g(\alpha)$ is a root of $h(x)$ , we know that it must satisfy $$ 0 = h(g(\alpha)) = g(\alpha)^2 + (b + \alpha) g(\alpha) +(\alpha^2 + b\alpha + c)$$ \begin{align} 0 &= r^2 \alpha^4 + (2rs+r)\alpha^3 + (s^2+2rt+br+s+1)\alpha^2 + \\ &\qquad +(2st+bs+t+b) \alpha + (t^2+bt+c) \end{align} Remembering that $f(\alpha) =0$, that is $$ \alpha^3 = -b\alpha^2-c\alpha-d$$ from which it follows that $$ \alpha^4 = -b\alpha^3-c\alpha^2-d\alpha = (b^2-c)\alpha^2+(bc-d)\alpha + bd$$ that condition can be written as \begin{align} 0 &= (r^2b^2-r^2c-2rsb+s^2+2rt+s+1)\alpha^2 +\\ &\qquad + (r^2bc-r^2d-2rsc-rc+2st+bs+t+b) \alpha + \\ &\qquad + (r^2bd-2rsd-rd + t^2+bt+c) \end{align} If we want this to be true no matter which root $\alpha$ of $f(x)$ we start with, we need \begin{align} 0 &= r^2b^2-r^2c-2rsb+s^2+2rt+s+1\\ 0&= r^2bc-r^2d-2rsc-rc+2st+bs+t+b \\ 0&= r^2bd-2rsd-rd + t^2+bt+c \end{align} which is a set of second-degree polynomial equations on $r$,$s$, $t$ with $b$, $c$, $d$ being known parameters. It is complicated, but it has some solution (sometimes in complex numbers). For $f(x)=x^3-3x+1$, that is for $b=0$, $c=-3$, $d=1$, it can be checked that $r=1$, $s=0$, $t=-2$ is one of the solutions, and this solution gives you $g(x)=x^2-2$ from your question. Either way, if we find a solution to these equations we find a polynomial $g(x)$ which transforms each root of $f(x)$ into another.
Since this set of equations can have many solutions, we may get various polynomials $g(x)$, some of which may produce roots in a cyclic way, while others don't. For example, for $f(x)=x^3-3x^2+2x$ (with roots $0$, $1$ and $2$) we can find that $g(x)=x^2-3x+2$ satisfies these equations, but doesn't transform the roots in a cyclic way (we have $g(0)=2$, $g(1)=g(2)=0$). There are five other solutions that transform the roots in non-cyclic way, but there are also two solutions that do that in a cyclic way: $g(x)=-\frac32x^2+\frac52x+1$ and $g(x)=\frac32x^2-\frac72 x+2$. My conjecture is that for any cubic polynomial with three distinct roots there will always be 8 solutions of the equations I derived, two of which will transform the roots in a cyclic way (in two directions). For example the polynomial that rotates the roots of $f(x)=x^3-3x+1$ in the oposite way to $x^2-2$ is $g(x)=-x^2-x+2$.
This method should work for polynomials $f$ of any degree $n$, though it quickly grows in complexity and in general it requires to solve a set of $n$ polynomial equations of degree $n-1$.