we generally know that if we have some matrix $A$ and we diagonalize it and get a matrix $D$, there is some matrix $P$ such that:
$$ D = P^{-1}AP$$
My question is then, does the matrix P generally contain the eigenvectors corresponding to the eigenvalues on the diagonal of D? If yes, why? I fail to see how that comes out explicitly, but it seems it is that way (is it true only for orthogonal matrices?).
Thank you!
That is indeed true. If
$$A = PDP^{-1}$$
is a diagonalization of $A$, $P$ is a matrix whose column vectors are eiegenvectors corresponding to the diagonal entries in $D$, which would then be the eigenvalues. How can you see this? First let $P = [v_1 \quad v_2 \quad \cdots \quad v_n]$ and assume that the diagonal elements of $D$ are $\lambda_1, \lambda_2, \ldots, \lambda_n$. Observe that
$$PD = [v_1 \quad v_2 \quad \cdots \quad v_n]D = [\lambda_1 v_1 \quad \lambda v_2 \quad \cdots \quad \lambda_n v_n] $$
$$AP = A[v_1 \quad v_2 \quad \cdots \quad v_n] = [Av_1 \quad Av_2 \quad \cdots \quad Av_n]$$
Since $AP = DP$ by assumption, we easily see that the $v_i$-s are eigenvectors of $A$ with corresponding eigenvalues $\lambda_i$-s.