About integral binary quadratic forms fixed by $\operatorname{GL_2(\mathbb Z)}$ matrices of order $3$

Question

I am reading this paper of Manjul Bhargava and Ariel Shnidman, and I want to prove this claim, which appear at the first paragraph of Theorem $14$:

Up to $\operatorname{SL_2}(\mathbb Z)$ equivalence and scaling, there is only one integral binary quadratic form having an $\operatorname{SL_2}(\mathbb Z)$-automorphism of order three, namely $Q(x,y)=x^2+xy+y^2$.

Now the pertinent definitions in order to understand the problem.

Consider the following action of $\operatorname{GL_2}(\mathbb Z)$ on the set of integral binary quadratic forms: for $\gamma\in$$\operatorname{GL_2}(\mathbb Z)$ and $f(x,y)=Px^2+Qxy+Ry^2$, with $P,Q,R\in\mathbb Z$, we define $(\gamma f)(x,y)=f\bigl((x,y)\gamma\bigr)$. Identifying each such $f$ with the triplet $(P,Q,R)^t$, we have that the triplet associated to $\gamma f$ is given by the matricial product

$$M_{\gamma}\,\cdot\begin{bmatrix} P\\ Q\\ R\end{bmatrix},\ \text{where}\ \ \gamma=\begin{bmatrix} A & B\\ C & D \end{bmatrix}\,\ \text{and}\ M_{\gamma}=\begin{bmatrix} A^2 & AB & B^2\\ 2AC & AD+BC & 2BD\\ C^2 & CD & D^2 \end{bmatrix}.$$

With these notations the authors' claim can be restated as follows:

Suppose that $f=(P,Q,R)^t$ is fixed by $\gamma\in\operatorname{GL_2}(\mathbb Z)$ of order $3$ and $Q\ne0$. Show that there exist $\theta\in\operatorname{GL_2}(\mathbb Z)$ and $n\in\mathbb Z$ such that $\theta f=(n,n,n)^t.$

I have had a very bad time trying to prove this fact. Perhaps my understanding of the problem is wrong, but I honestly don't think so (I successfully completed the previous details of the paper).

My work so far: Let $a=A+D$ be the trace of $\gamma$. Since $\gamma^3=I$ then $\det\gamma=1$, so the characteristic polynomial of $\gamma$ is $X^2-aX+1$. Now

$$X^3-1=(X+a)(X^2-aX+1)+(a+1)\bigl[(a-1)X-1\bigr]\,,$$

and since $\gamma$ also satisfies $\gamma^3-I=0$, then after evaluating the equality above at $\gamma$ we obtain that either $a=-1$ or $(a-1)\gamma=I$; but $(a-1)\gamma=I$ implies $(a-1)^3=1$, that is $a=2$, and so $I=(a-1)\gamma=\gamma$, contradicting the fact that $\gamma$ has order $3$ in $\operatorname{GL_2}(\mathbb Z)$.

On the other hand $\gamma^{-1}$ also fixes $f$, so denoting $(P,Q,R)^t$ by $v$ we obtain $(M_{\gamma^{-1}}-M_\gamma)v=0$. Using the formula $\gamma^{-1}=\binom{\ \ D\ \ -B}{\!\!\!-C\ \ \ \ A}$ together with the equalities $AD-BC=1$ and $A+D=-1$ we get

$$\begin{align*} M_{\gamma^{-1}}-M_{\gamma}=&\,\begin{bmatrix} D^2 & -BD & B^2\\ -2CD & AD+BC & -2AB\\ C^2 & -AC & A^2 \end{bmatrix}-\begin{bmatrix} A^2 & AB & B^2\\ 2AC & AD+BC & 2BD\\ C^2 & CD & D^2 \end{bmatrix} \\[5mm] =&\,\begin{bmatrix} A-D & B & 0 \\ 2C & 0 & 2B\\ 0 & C & D-A \end{bmatrix}\,, \end{align*}$$

and so $v$ has the specific form $v=\frac Q{A-D}(-B,A-D,C)^t$.

Now I am stuck at this point: If $\theta=\binom{B\ \ \ 0}{D\ \ \ 1}$ then $M_\theta\cdot(n,n,n)^t=v$, where $n=\frac{-Q}{B(A-D)}$ (it is easy to see from the equalities $AD-BC=1$ and $A+D=-1$ that $B(A-D)\ne0$). The problem is, of course, that $n$ is not necessarily an integer and that $\theta$ not necessarily belong to $\operatorname{GL_2}(\mathbb Z)$. Changing $B$ by $1$ in $\theta$ does not work, because scaling the first row of $\theta$ amounts to a corresponding quadratic scaling at the first row of $M_\theta$, but keeping the same scaling at the second row of $M_\theta$, so the desired equality is not preserved.

I tried every possible matrix obtained from $\gamma$ with no success, so any help is welcomed!!!

Answer 1

The modular group $SL_2 \mathbb Z$ is presented by $$ \langle T,P | T^4 = P^3 = 1 \rangle, $$ where we mean specific matrices $$ T = \left( \begin{array}{rr} 0 & -1 \\ 1 & 0 \end{array} \right) $$ and $$ P = \left( \begin{array}{rr} 0 & 1 \\ -1 & -1 \end{array} \right) $$ This is from page 9 in Binary Quadratic Forms by Duncan A. Buell. Proof takes a few pages.

Now, $P$ really is in the automorphism group for $x^2 + xy + y^2,$ since $$ \left( \begin{array}{rr} 0 & -1 \\ 1 & -1 \end{array} \right) \left( \begin{array}{rr} 2 & 1 \\ 1 & 2 \end{array} \right) \left( \begin{array}{rr} 0 & 1 \\ -1 & -1 \end{array} \right)= \left( \begin{array}{rr} 2 & 1 \\ 1 & 2 \end{array} \right) $$

Next, I believe already (confirmed by studiosus, comment after answer as well as a separate answer) that any group element of order 3 is conjugate to $P$ or $P^2,$ of the form $gP g^{-1}$ for example. But then we simply absorb $g$ into the Gram/Hessian matrix for the form; that is, if $$ (g^{-1})^TP^T g^T H gP g^{-1} = H, $$ then $$ P^T g^T H g P = g^T H g, $$ we rename $$ G = g^T H g $$ to arrive at $$ P^T G P = G. $$

Note that $g^T$ and $g^{-1}$ are not quite the same, and the expression $g^T H g$ is the correct expression for the Gram matrix of a new binary quadratic form; this new form is actually called "equivalent" to the original.

Finally, what things have $P$ in the automorphism group? $$ \left( \begin{array}{rr} 0 & -1 \\ 1 & -1 \end{array} \right) \left( \begin{array}{rr} 2A & B \\ B & 2C \end{array} \right) \left( \begin{array}{rr} 0 & 1 \\ -1 & -1 \end{array} \right)= \left( \begin{array}{cc} 2C & 2C-B \\ 2C-B & 2A-2B+2C \end{array} \right) $$ which give the equations $$ 2A = 2C, \; \; 2C-B = B, \; \; 2A-2B+2C = 2C, $$ so actually $A=B=C.$

As far as I can tell that is it. It would be nice if someone with a more sure hand with generators and relations confirmed, or corrected, the part about elements of order 3 in the group.

Answer 2

It is a bit too long for a comment, it provides details for Will's answer: Consider a group $G$ which splits as a free product $G_1 * G_2$.

Theorem. Every finite order element is conjugate either into $G_1$ or into $G_2$.

Proof. The trick is to think of this not as an algebraic but a geometric problem. The free product decomposition of $G$ corresponds to a simplicial action of $G$ on a Bass-Serre tree $T$, which is a certain simplicial tree such that the vertex stabilizers are conjugates of $G_1$ and $G_2$ (there are exactly two $G$-orbits of vertices), and the edge-stabilizers are trivial. The construction of this tree is not hard at all, especially if you are a topologist familiar with van Kampen theorem (and Serre does not assume any knowledge of topology and proves everything from scratch by imitating Tits' BN pairs theory). As a topologist, I would consider a wedge $K$ of $K_1=K(G_1,1)$ and $K_2=K(G_2,1)$ (call $e$ the point where they are joined together) and look at its universal cover. Lifts of $K_1$ and $K_2$ are the vertices of the tree and lifts of $e$ are the edges. By van Kampen's theorem, $\pi_1(K)\cong G_1* G_2$. Everything else is elementary.

Now, if you have any finite-order element $g\in G$, its orbit in the vertex set of $T$ is finite and, hence, spans a finite subtree. It is an elementary argument nicely explained by Serre in his book that each group acting on a finite tree $S$ has a fixed point. (Just keep removing all leaves of the tree $S$ inductively until you have a fixed vertex left.) Therefore, such $g$ is conjugate into one of the subgroups $G_1$ or $G_2$.