Let $K$ be a field and let $X ∈ K^{n×n}$ be a matrix. Show that the set
$$U := \{A \in GL_n(K) | A^ TXA = X\}$$
is a group with respect to matrix multiplication.
For this exercise, associativity is pretty self explanatory since by nature matrices are associative. I'm confused about how to show that it has an inverse . For Identity, I think I would go about $A^ TXAI = XI=X$ but I'm not sure if it is correct.
Observe first that
$I^TXI = IXI = X \Longrightarrow I \in U; \tag 1$
note that
$AA^{-1} = I \Longrightarrow (A^{-1})^TA^T = I^T = I \Longrightarrow (A^{-1})^T = (A^T)^{-1}; \tag 2$
in light of this, we further observe that
$A \in U \Longrightarrow A^TXA = X \Longrightarrow$ $ (A^T)^{-1}XA^{-1} = X \Longrightarrow (A^{-1})^TXA^{-1} = X \Longrightarrow A^{-1} \in U ; \tag 3$
finally, observe that
$A, B \in U \Longrightarrow A^TXA = X, \; B^TXB = X \Longrightarrow$ $(AB)^TX(AB) = B^TA^TXAB = B^TXB = X \Longrightarrow AB \in U. \tag 4$
We see that $U$ contains the identity element $I$, the inverse $A^{-1}$ of any $A \in U$, and is closed under multiplication; multiplication in $U$ is associative since it is inherited from $GL_n(K)$; thus $U$ is a subgroup of $GL_n(K)$.
In closing, we note that one may also follow the suggestion made by Dave in the comments to the question itself, and show that
$A \in U, B \in U \Longrightarrow AB^{-1} \in U, \tag 5$
which also implies $U$ is a group, e.g.
$I = II^{-1} \in U, \tag 6$
and so forth; but showing that (5) implies $U$ is a group is about as much work as (1)-(4) combined, so not much effort is ultimately saved, though the point of view is of some value in many problems.