Let $ Q\in \mathbb{Z}[x_1, \dots, x_n ] $ be an integer quadratic form. That is, $ Q $ is a homogeneous degree $ 2 $ polynomial with integer coefficients. Is there a good way to determine if $ Q $ has any integer roots (other than all $ 0 $)?
I don't know if this helps but in the example I'm thinking about there are about $ 20 $ variables and the coefficients are very small, $ < 10 $.
For context I have a set $ \mathcal{E} $ of 15 orthogonal matrices with integer entries and I'm trying to find integer zeroes of the quadratic form $ Q $: $$ Q(x_{ij}):= \sum_{E \in \mathcal{E}}(\text{Tr}(EX))^2−\sum_{E \in \mathcal{E}}\text{Tr}(EXE^{−1}X) $$ where the variables I am solving for are the entries of the symmetric matrix $ X $, and $\text{Tr}$ is the trace.
I would even be satisfied if I could determine if $ Q $ has real roots. But integer roots are preferable.
$ \newcommand\trans{^{\mathrm T}} \newcommand\span{\mathrm{span}} $You're looking for "isotropic elements of $Q$ in the module $\mathbb Z^n$". This is the same as finding roots over the vector space $\mathbb Q^n$ because if you have such a root $x$ then $ax \in \mathbb Z^n$ for some $a \in \mathbb Z$.
I will start with the real case because it is easier, and then explain what changes with $\mathbb Q$.
For existence of real roots you just have to check that $Q$ is not positive- or negative-definite.
To find real roots, you can diagonalize: form the symmetric matrix $M$ such that $x\trans Mx = Q(x)$ with $x = (x_1,\dotsc,x_n)\trans$ and diagonalize $M = V\trans DV$. If $e_i = (0,\dotsc,1,\dotsc,0)\trans$ with $1$ in the $i^\text{th}$ position (so that $\{e_i\}_{i=1}^n$ forms the standard basis of $\mathbb R^n$) then $f_i = V^Te_i$ is an orthogonal basis for $Q$. By construction $Q(f_i) = D_{ii}$. The signature $(p, q, r)$ of $Q$ is the number of $f_i$ such that $Q(f_i) > 0$, $Q(f_i) < 0$, and $Q(f_i) = 0$, respectively.
We could also start with a basis of $\mathbb R^n$ and apply Gram-Schmidt, but there are edge cases to consider when $Q$ is not definite.
Let $a_1,\dotsc,a_p$ be those $f_i$ with $Q(f_i) > 0$, let $b_1,\dotsc,b_q$ those with $Q(f_i) < 0$, and $c_1,\dotsc,c_r$ those with $Q(f_i) = 0$. Also normalize $a_1,\dotsc,a_p$ so that $Q(a_i) = 1$ and normalize $b_1,\dotsc,b_q$ so that $Q(b_i) = -1$.
If $r \ne 0$ then all linear combinations of $c_1,\dotsc,c_r$ give a root of $Q$. These are called radical vectors. However, I can easily imagine you constructed $Q$ such that $r = 0$ and there are no radical vectors. From here on we assume $r = 0$.
From here, every vector $v$ is of the form $$ v = \alpha_1a_1 + \dotsb + \alpha_qa_p + \beta_1b_1 + \dotsb + \beta_qb_q $$ and $v$ is isotopic iff $$ \sum_{i=1}^p\alpha_i^2 - \sum_{j=1}^q\beta_j^2 = 0. $$ But we can say some more.
The quantity $w = \min\{p, q\}$ is precisely the Witt index of $Q$. This is the dimension of a maximal totally isotropic subspace, i.e. a subspace of maximal dimension where all vectors are isotropic and so roots of $Q$. If $w = 0$ then $q = 0$ or $p = 0$ and $Q$ is positive- or negative-definite and has no roots.
If $w \ne 0$, then we can construct two such maximal totally isotropic subspace $W, W'$ by pairing up basis vectors. Define $\nu_i = a_i + b_i$ and $\eta_i = a_i - b_i$ for (say) $i = 1,\dotsc,w$. Then $Q(\nu_i) = Q(\eta_i) = 0$ and we can take $$ W = \span\{\nu_1,\dotsc,\nu_w\},\quad W' = \span\{\eta_1,\dotsc,\eta_w\}. $$ These subspaces are complementary in the sense that we can construct $W'$ knowing just $W$ and vice versa. The space $H = W \oplus W'$ is called a hyperbolic subspace. Finding more isotropic vectors in $H$ is easy; every vector $v \in H$ may be written $$ v = \alpha_1\nu_1 + \dotsb + \alpha_w\nu_w + \beta_1\eta_1 + \dotsb + \beta_w\eta_w $$ where $\alpha_i, \beta_i \in \mathbb R$. If every pair $(\alpha_i,\beta_i)$ has $\alpha_i = 0$ or $\beta_i = 0$ then $v$ is isotropic; more generally $v$ is isotropic iff $\sum_i\alpha_i\beta_i = 0$.
If $p \ne q$ then we still have some $a_{w+1},\dotsc,a_p$ or $b_{w+1},\dotsc,b_q$ remaining. Let $m = \max\{p,q\}$ and call these $\xi_1,\dotsc,\xi_{m-w}$. These span a subspace $A = \span\{\xi_1,\dotsc,\xi_{m-w}\}$ which is anisotropic, meaning there are no isotropic vectors. Any of $\xi_1,\dotsc,\xi_{m-q}$ can replace $a$ or $b$ (whichever is appropriate) in the construction of $W, W'$ to get other maximal totally isotropic subspaces.
Returning briefly to $r$ not necessarily $0$ and defining $R = \span\{c_1,\dotsc,c_r\}$, the decomposition $$ \mathbb R^n = A\oplus H\oplus R $$ is called a Witt decomposition of $\mathbb R^n$, and the dimensions of each piece are invariants of $Q$.
To be clear, not all isotropic vectors lie in $W$ or $W'$, this is just a convenient way of constructing many of them.
When we consider $\mathbb Q^n$ instead of $\mathbb R^n$, much of this is the same. There are two potential difficulties I see: