Find a basis of $\mathbb{R}^4$ for which the representation of $$F(x,y) = x^T\begin{bmatrix} 1 & 1 & 2 & 0 \\ 1 & 0 & 1 & -1 \\ 2 & 1 & 3 & -1 \\ 0 & -1 & -1 & -1 \\ \end{bmatrix} y$$ is diagonal, and write the matrix of $F$ with respect to this basis (Use row and column operations).
I think by converting the matrix into it's quadratic form and by using Sylvester's Law of Inertia or completing the square method, we can find the basis by orthogonalization-like process. But I have no idea how the basis can be found by using row and column operations. I saw some similar questions here but most of them are solved by the use of Eigen-values and since we don't know that topic yet, there is probably a different solution method. Any hint or help would be appreciated. Thank you in advance.
Let's see, it comes out that the original polynomial, let us write it in variables $w,x,y,z,$ is a difference of squares, meaning it also factors as the product of two linear forms. This is sufficiently unusual (with everything integers) that I would bet the question was constructed using the factoring.
$$ (w+x+2y)^2 - (x+y+z)^2 = \; \; (w + 2x + 3y + z)(w+y-z) $$ Notice how the product makes obvious the $3y^2,$ also $-z^2,$ but zero $x^2$ terms.
$$ P^T H P = D $$ $$\left( \begin{array}{rrrr} 1 & 0 & 0 & 0 \\ - 1 & 1 & 0 & 0 \\ - 1 & - 1 & 1 & 0 \\ 1 & - 1 & 0 & 1 \\ \end{array} \right) \left( \begin{array}{rrrr} 1 & 1 & 2 & 0 \\ 1 & 0 & 1 & - 1 \\ 2 & 1 & 3 & - 1 \\ 0 & - 1 & - 1 & - 1 \\ \end{array} \right) \left( \begin{array}{rrrr} 1 & - 1 & - 1 & 1 \\ 0 & 1 & - 1 & - 1 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array} \right) = \left( \begin{array}{rrrr} 1 & 0 & 0 & 0 \\ 0 & - 1 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ \end{array} \right) $$ $$ Q^T D Q = H $$ $$\left( \begin{array}{rrrr} 1 & 0 & 0 & 0 \\ 1 & 1 & 0 & 0 \\ 2 & 1 & 1 & 0 \\ 0 & 1 & 0 & 1 \\ \end{array} \right) \left( \begin{array}{rrrr} 1 & 0 & 0 & 0 \\ 0 & - 1 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ \end{array} \right) \left( \begin{array}{rrrr} 1 & 1 & 2 & 0 \\ 0 & 1 & 1 & 1 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array} \right) = \left( \begin{array}{rrrr} 1 & 1 & 2 & 0 \\ 1 & 0 & 1 & - 1 \\ 2 & 1 & 3 & - 1 \\ 0 & - 1 & - 1 & - 1 \\ \end{array} \right) $$
====================================================
algorithm: see http://math.stackexchange.com/questions/1388421/reference-for-linear-algebra-books-that-teach-reverse-hermite-method-for-symmetr
$$ H = \left( \begin{array}{rrrr} 1 & 1 & 2 & 0 \\ 1 & 0 & 1 & - 1 \\ 2 & 1 & 3 & - 1 \\ 0 & - 1 & - 1 & - 1 \\ \end{array} \right) $$
$$ D_0 = H $$ $$ E_j^T D_{j-1} E_j = D_j $$ $$ P_{j-1} E_j = P_j $$ $$ E_j^{-1} Q_{j-1} = Q_j $$ $$ P_j Q_j = I $$ $$ P_j^T H P_j = D_j $$ $$ Q_j^T D_j Q_j = H $$
$$ H = \left( \begin{array}{rrrr} 1 & 1 & 2 & 0 \\ 1 & 0 & 1 & - 1 \\ 2 & 1 & 3 & - 1 \\ 0 & - 1 & - 1 & - 1 \\ \end{array} \right) $$
==============================================
$$ E_{1} = \left( \begin{array}{rrrr} 1 & - 1 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array} \right) $$ $$ P_{1} = \left( \begin{array}{rrrr} 1 & - 1 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array} \right) , \; \; \; Q_{1} = \left( \begin{array}{rrrr} 1 & 1 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array} \right) , \; \; \; D_{1} = \left( \begin{array}{rrrr} 1 & 0 & 2 & 0 \\ 0 & - 1 & - 1 & - 1 \\ 2 & - 1 & 3 & - 1 \\ 0 & - 1 & - 1 & - 1 \\ \end{array} \right) $$
==============================================
$$ E_{2} = \left( \begin{array}{rrrr} 1 & 0 & - 2 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array} \right) $$ $$ P_{2} = \left( \begin{array}{rrrr} 1 & - 1 & - 2 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array} \right) , \; \; \; Q_{2} = \left( \begin{array}{rrrr} 1 & 1 & 2 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array} \right) , \; \; \; D_{2} = \left( \begin{array}{rrrr} 1 & 0 & 0 & 0 \\ 0 & - 1 & - 1 & - 1 \\ 0 & - 1 & - 1 & - 1 \\ 0 & - 1 & - 1 & - 1 \\ \end{array} \right) $$
==============================================
$$ E_{3} = \left( \begin{array}{rrrr} 1 & 0 & 0 & 0 \\ 0 & 1 & - 1 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array} \right) $$ $$ P_{3} = \left( \begin{array}{rrrr} 1 & - 1 & - 1 & 0 \\ 0 & 1 & - 1 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array} \right) , \; \; \; Q_{3} = \left( \begin{array}{rrrr} 1 & 1 & 2 & 0 \\ 0 & 1 & 1 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array} \right) , \; \; \; D_{3} = \left( \begin{array}{rrrr} 1 & 0 & 0 & 0 \\ 0 & - 1 & 0 & - 1 \\ 0 & 0 & 0 & 0 \\ 0 & - 1 & 0 & - 1 \\ \end{array} \right) $$
==============================================
$$ E_{4} = \left( \begin{array}{rrrr} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & - 1 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array} \right) $$ $$ P_{4} = \left( \begin{array}{rrrr} 1 & - 1 & - 1 & 1 \\ 0 & 1 & - 1 & - 1 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array} \right) , \; \; \; Q_{4} = \left( \begin{array}{rrrr} 1 & 1 & 2 & 0 \\ 0 & 1 & 1 & 1 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array} \right) , \; \; \; D_{4} = \left( \begin{array}{rrrr} 1 & 0 & 0 & 0 \\ 0 & - 1 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ \end{array} \right) $$
==============================================
$$ P^T H P = D $$ $$\left( \begin{array}{rrrr} 1 & 0 & 0 & 0 \\ - 1 & 1 & 0 & 0 \\ - 1 & - 1 & 1 & 0 \\ 1 & - 1 & 0 & 1 \\ \end{array} \right) \left( \begin{array}{rrrr} 1 & 1 & 2 & 0 \\ 1 & 0 & 1 & - 1 \\ 2 & 1 & 3 & - 1 \\ 0 & - 1 & - 1 & - 1 \\ \end{array} \right) \left( \begin{array}{rrrr} 1 & - 1 & - 1 & 1 \\ 0 & 1 & - 1 & - 1 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array} \right) = \left( \begin{array}{rrrr} 1 & 0 & 0 & 0 \\ 0 & - 1 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ \end{array} \right) $$ $$ Q^T D Q = H $$ $$\left( \begin{array}{rrrr} 1 & 0 & 0 & 0 \\ 1 & 1 & 0 & 0 \\ 2 & 1 & 1 & 0 \\ 0 & 1 & 0 & 1 \\ \end{array} \right) \left( \begin{array}{rrrr} 1 & 0 & 0 & 0 \\ 0 & - 1 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ \end{array} \right) \left( \begin{array}{rrrr} 1 & 1 & 2 & 0 \\ 0 & 1 & 1 & 1 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array} \right) = \left( \begin{array}{rrrr} 1 & 1 & 2 & 0 \\ 1 & 0 & 1 & - 1 \\ 2 & 1 & 3 & - 1 \\ 0 & - 1 & - 1 & - 1 \\ \end{array} \right) $$