It is quite easy to use the LLL algorithm to find approximate solutions to linear/multilinear forms, and I am able to do that. However, I am trying to understand how the LLL algorithm is used to approximate solutions to quadratic forms, which is used to find rational approximations.
In this small excerpt, the paper uses the LLL algorithm to find rational approximations for real numbers(as one of the applications of the LLL algorithm) by representing the approximation as a quadratic form.
Since $x$ and $y$ are commutative, the expansion of $q(x,y)$ is
$$\left(M\bar{\alpha}^2+\frac{1}{M}\right)x^2+My^2-2M\bar{\alpha}xy$$
Here the coefficient of $xy$ is $-2M\bar{\alpha}$. However, the paper uses $-M\bar{\alpha}$, which I'm not quite sure why. Furthermore I've tried using the LLL algorithm on this lattice, but it failed to find rational approximations
Using a slightly modified matrix - $$\begin{pmatrix}\bar{a}^2M+1/M&-2\bar{a}M\\-2\bar{a}M&M\end{pmatrix}$$
The shortest vector found is still quite a poor rational approximation of $\bar{a}/2$.
So how could I use LLL on a quadratic form, or more generally any polynomial to find small roots?
Paper: https://simond.users.lmno.cnrs.fr/maths/lll25_Simon.pdf
Appendix
Let $\bar{a}=3.14159265358979\approx\pi$, $M=10^{10}$
$$\begin{pmatrix}\bar{a}^2M+1/M&-\bar{a}M\\-\bar{a}M&M\end{pmatrix}$$ After LLL: $$\begin{pmatrix}37590&69064\\-174394&87912\end{pmatrix}$$ which are far from approximating $\bar{a}$
Using the 'modified' matrix,
$$\begin{pmatrix}\bar{a}^2M+1/M&-2\bar{a}M\\-2\bar{a}M&M\end{pmatrix}$$ After LLL: $$\begin{pmatrix}-26967662134&-42831853072\\-6283185307&10000000000\end{pmatrix}$$ which kind of approximate $\frac{\bar{a}}{2}$ ($2\times\frac{42831853072}{26967662134}\approx3.17653438842212$, albeit very poorly) which are far from approximating $\bar{a}$
Alternatively, I've tried approximating using a linear form(which I'm familiar with using LLL on):
Consider $$x\bar{\alpha}-y=0$$
Construct the LLL matrix as
$$\begin{pmatrix}1&0&aM\\0&1&M\end{pmatrix}$$ After LLL, $$\begin{pmatrix}33102&103993&-3940\\113&355&-302110\end{pmatrix}$$
Which indeed gives 2 approximations for $\pi$, $\frac{103993}{33102}=[3; 7, 15, 1, 292]$ and $\frac{355}{113}=[3; 7, 15, 1]$.