Consider a matrix $$ A=\begin{bmatrix} 9&-1&-2\\ -1&9&-2\\ -2&-2&6\end{bmatrix}. $$ We now find the orthogonal matrix $P$ such that $P^TAP$ is diagonal.
My attempt:
Eigenvalues are $4$ and $10$ (with multiplicity $2$). Eigenvectors corresponding to $4$ are $~c \begin{bmatrix} 1\\1\\2\end{bmatrix},~c \neq 0~$ and corresponding to $10~$ are $~c \begin{bmatrix} 1\\0\\-\dfrac{1}{2}\end{bmatrix}+ d\begin{bmatrix} 0\\1\\-\dfrac{1}{2}\end{bmatrix},~c,d \neq 0.$ Now from here how can I get an orthogonal matrix so that $~P^TAP~$ is diagonal? If the question is about invertible matrix, the we choose $~P=\begin{bmatrix} 1&1&0\\0&1&1\\ 2&-\frac{1}{2}& -\frac{1}{2}\end{bmatrix}.$ But how can I get an orthogonal matrix from here?
Please help me to solve this.
As it is $3\times 3$ one can use this trick.
Take a $4$-eigenvector, $a:=(1,2,2)$.
Take any $10$-eigenvector, $b:=(2,0,-1)$ say.
These are bound to be orthogonal to each other.
They are also orthogonal to $a\times b=(-1,5,-2)$, and that is bound to be your third vector.
Now all you need to do is normalise these.