Let
$$A = \begin{bmatrix} -2 & 7 \\ 5 &3 \end{bmatrix}$$
Using the triangularization theorem, I want to find a matrix $P$ such that $P^{T}A P$ is an upper triangular matrix.
I have the answer
$$P = \begin{bmatrix} \frac{6}{37} \sqrt{38}&\frac{1}{37} \sqrt{52} \\ \frac{-1}{37} \sqrt{37} & \frac{6}{37} \sqrt{37}\end{bmatrix}$$
but I am very confused on how to get this matrix $P$. Does anyone know the process? Thank you.
We have $AP=PT$
In particular, we have $$AP_1 = T_{11}P_1$$ $$AP_2 = T_{12}P_1 + T_{22}P_2 = P\begin{bmatrix} T_{12} \\ T_{22}\end{bmatrix}$$
After picking $P_1$ to be an eigenvector with unit length, just pick $P_2$ to be orthogonal to $P_1$.
We then compute $$\begin{bmatrix} T_{12} \\ T_{22}\end{bmatrix} = P^TAP_2$$
Extra exercises: Think of how many possible $P$'s you can pick for this question.