Sometimes I come across exam tasks that basically ask me to "Find the Matrix Group that preserves (or is isomorphic to a Group that preserves) a given function from a vector space to a field". Usually the answer is a "named" Group, i.e., one of the standard ones (orthogonal, unitary, symplectic...) or a tensor product of more of them.
For instance I am asked to find the Group that preserves (in its action on $\mathbb{R}^3$) the $\mathbb{R}^3\rightarrow \mathbb{R}$ function
$$2x_1x_2+x_3^2$$
where $x_1, x_2, x_3$ are the coordinates of a vector in $\mathbb{R}^3$. I believe this is equivalent to the Group $G$ of matrices $g$ such that $$g^T A g=A\qquad \forall g\in G$$ where $$A=\begin{pmatrix} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{pmatrix}$$ However, I cannot find any better way to describe this group, nor can I find a relation with any of the usual Matrix Groups. If the problem was bidimensional I would try to brutally calculate the properties of the Matrix elements of $g$, but in $d\geq 3$ this becomes tedious, there has to be a clever way. How would you approach such a problem in general?
One thing you can do in such a case is (try to) diagonalize the matrix $A$. In this case for example, the matrix is congruent to $$D=\begin{pmatrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 &0 & -1\end{pmatrix},$$ so you can find an orthogonal matrix $P$ such that $D=PAP^{T}$. Then if $g$ is such that $g^TAg=A$, conjugating both sides by $P$ will give you $$D=PAP^T=(Pg^TP^T)(PAP^T)(PgP^T)=(PgP^T)^TD(PgP^T).$$ In particular, this tells you that $G$ is conjugated (hence isomorphic) to the orthogonal group $O(2,1)$.