Let $G$ be a finite group with elements $g_1, g_2, \ldots, g_n$. We define the group matrix by $$X_G = [x_{g_ig_j^{-1}}].$$
We then can define the group determinant as $$\det X_G = \Theta_G.$$
$\Theta_G$ will be a polynomial over the $x_{g_k}$. It seems like the group determinant would be independent (perhaps just up to sign $(\pm)$) from the indexing of the elements of $G$ since any renumbering of the elements would just cause row and column swaps.
Is this a proper way to think of this?
The determinant is indeed independent from the indexing of the elements.
One way to see this is the following. Suppose you index your elements in a different way; this means you take an element $\sigma$ of the symmetric group $S_n$, and you order your element as $g_{\sigma(1)}, g_{\sigma(2)}, \ldots, g_{\sigma(n)}$. With this new ordering, the matrix becomes $$X^\sigma_G=[x_{g_{\sigma(i)}g_{\sigma(j)}^{-1}}]. $$
Now, let $P$ be the permutation matrix associated to $\sigma$, that is, $P=[\delta_{\sigma(i), j}]$, where $\delta_{k, \ell}$ is $1$ if $k=\ell$ and $0$ else. If $M$ is any $n\times n$ matrix, then the matrix $PM$ is obtained by reordering the rows of $M$ by applying $\sigma$; similarly, $MP^{-1}$ is obtained by reordering the columns of $M$ by applying $\sigma$. Combining these two observations, you get that $X^\sigma_G = PX_G P^{-1}$. Taking the determinant, you have $det(X^\sigma_G) = det(X_G)$.
Note that the determinants are exaclty equal, not up to a sign.