I see different ways to diagonalize a matrix.
sometimes they use the inverse like: $A = P^T D P$ and sometimes they use transverse like: $A = P^{-1} D P$
I've read that when a matrix is orthogonal, then $ Q^T = Q^{-1}$
$P$ should be the eigenvectors of matrix $A$, but also sometimes they make $P$ orthonormal to each other. I also don't know when and why would you do that.
Thanks in advance,
It is always the inverse.
Diagonalisation is about changing basis so that the transformation represented by $A$ becomes diagonal (and represented by $D$ instead). And when changing basis, the matrix representation of linear transformations is always sandwiched between the change-of-basis matrix $P$ and its inverse $P^{-1}$ (in the appropriate order).
However, if the transformation / matrix in question is symmetric, then one can take the change-of-basis matrix to be orthogonal. And in those cases, one might instead write its inverse as $P^T$, using transpose notation, to make it clear that this is a special case (and also make it clear how easy the inverse is to calculate in this case).