$AA^t=BB^t \implies A=B$

Question

Let $A$ and $B$ are complex valued $n\times n$ matrices and let $\det(A)=1$ or $-1$.

Then what necessary and sufficient conditions can be given for $A$ and $B$ to satisfy this proposition?

"If $AA^t=BB^t$, then $A=B$ or $A=-B$." where $A^t$ is the transpose of $A$.

Answer 1

Warning and Notes: I wrote the answer for real matrices and transpose $(\cdot)^t$. One can simply adapt it for complex matrices and complex conjugate transpose. However, complex matrices and only transpose is a complete other story...

In the question it is not clear, if the condition is on $(A,B)$, that is some logical formula $P(A,B)$, or the same condition is on $A$ and $B$, that is some logical formula $P(A) \land P(B)$. For the second case, there is no "solution" for $n>1$ (cf @ErickWong's comment to the question).

Now the results...

Claim 1: The condition "$A,B$ are symmetric and definite" is sufficient to make $$ AA^t = BB^t \implies A=B \lor A=-B$$ true.

Proof: First assume $A$ is positive definite. Then, $ AA^t = BB^t $ implies $I = A^{-1}B (A^{-1}B)^t$, that is $M = A^{-1}B$ is orthogonal. But $M$ is similar to $RMR^{-1} = R^{-1}BR^{-1}$, which is definite. That is, $M$ is either $I$ (in case $B$ is positive definite), or $-I$ (in case $B$ is negative definite). But that means $A = B$ or $A = -B$.

Now, assume $A$ is negative definite. Then, $\tilde A = -A$ is positive definite. Thus, by previous paragraph, we have $-A = B$ or $-A = -B$.

Claim 2: Let $ M_0 = \{ A\in\mathbb R^{n\times n} : |\det A| = 1 \}, $ $ M_1 = \{ A\in M_0 : A \text{ symm. and definite} \}, $ and $M_2 \subseteq M_0$ containing $M_1$ with the property $\forall A,B\in M_2: [AA^t = BB^t \implies A = B \lor A=-B]$.

Then, $M_2 = M_1$.

Proof: Let $A\in M_2$. From $\tilde A = (AA^t)^{1/2}\in M_1 \subseteq M_2$ follows $A = \tilde A$ or $A = -\tilde A$. That is, $A$ is already in $M_1$. (thanks to @ErickWong)