Let $M \in \mathbb{R}^{n \times n}$ be a positive definite matrix. I would like to characterize the set $$ \mathcal{A} := \{A \in \mathbb{R}^{n\times n} : AMA' \text{ is diagonal and invertible} \}. $$
Clearly one element of $\mathcal{A}$ is the orthogonal matrix given by the eigenvectors of $M$ and one can multiply it by a non-zero scalar to generate other elements. However I don't know whether these are the only elements.
Any help willbe highly appreciated!
Here's one way you can think about it. The operation $\left<x,y\right>_M = x^TMy$ defines an inner product on $\mathbb{R}^n$. Then, the matrices $A$ such that $AMA'$ is diagonal will merely be the ones whose rows are orthogonal under this inner product.