Given the quadratic form $Q(\textbf{x}) = x^2 + 2xy - 4xz +2yz -4z^2$, the question asks to decompose $Q$ into sums of squares, first by eliminating terms in $z$, then terms in $y$, and then terms in $x$.
I know the signature of $Q$ is $(2,1)$ from the solution, but I'm at a loss for how to actually perform what the question is asking. For instance, I recognize that I can take the terms in $x$ and $z$ and form $(x-2z)^2$, but I don't really see what to do next. Any suggestions?
a method. Usually we take the matrix $H$ to be the Hessian matrix of second partial derivatives. In the special case of integer coefficients with all the mixed coefficent terms even, we can get $H$ all integers by taking half the Hessian matrix.
Outcome this time:
$$ P^T H P = D $$ $$\left( \begin{array}{rrr} 1 & 0 & 0 \\ - 1 & 1 & 0 \\ - 1 & 3 & 1 \\ \end{array} \right) \left( \begin{array}{rrr} 1 & 1 & - 2 \\ 1 & 0 & 1 \\ - 2 & 1 & - 4 \\ \end{array} \right) \left( \begin{array}{rrr} 1 & - 1 & - 1 \\ 0 & 1 & 3 \\ 0 & 0 & 1 \\ \end{array} \right) = \left( \begin{array}{rrr} 1 & 0 & 0 \\ 0 & - 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right) $$ $$ Q^T D Q = H $$ $$\left( \begin{array}{rrr} 1 & 0 & 0 \\ 1 & 1 & 0 \\ - 2 & - 3 & 1 \\ \end{array} \right) \left( \begin{array}{rrr} 1 & 0 & 0 \\ 0 & - 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right) \left( \begin{array}{rrr} 1 & 1 & - 2 \\ 0 & 1 & - 3 \\ 0 & 0 & 1 \\ \end{array} \right) = \left( \begin{array}{rrr} 1 & 1 & - 2 \\ 1 & 0 & 1 \\ - 2 & 1 & - 4 \\ \end{array} \right) $$
It is the diagonal elements of $D$ and the rows of $Q$ that give the expression, $$ \color{red}{ (x+y-2z)^2 - (y-3z)^2 + z^2 } $$
Algorithm discussed at http://math.stackexchange.com/questions/1388421/reference-for-linear-algebra-books-that-teach-reverse-hermite-method-for-symmetr
https://en.wikipedia.org/wiki/Sylvester%27s_law_of_inertia
$$ H = \left( \begin{array}{rrr} 1 & 1 & - 2 \\ 1 & 0 & 1 \\ - 2 & 1 & - 4 \\ \end{array} \right) $$ $$ D_0 = H $$ $$ E_j^T D_{j-1} E_j = D_j $$ $$ P_{j-1} E_j = P_j $$ $$ E_j^{-1} Q_{j-1} = Q_j $$ $$ P_j Q_j = Q_j P_j = I $$ $$ P_j^T H P_j = D_j $$ $$ Q_j^T D_j Q_j = H $$
$$ H = \left( \begin{array}{rrr} 1 & 1 & - 2 \\ 1 & 0 & 1 \\ - 2 & 1 & - 4 \\ \end{array} \right) $$
==============================================
$$ E_{1} = \left( \begin{array}{rrr} 1 & - 1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right) $$ $$ P_{1} = \left( \begin{array}{rrr} 1 & - 1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right) , \; \; \; Q_{1} = \left( \begin{array}{rrr} 1 & 1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right) , \; \; \; D_{1} = \left( \begin{array}{rrr} 1 & 0 & - 2 \\ 0 & - 1 & 3 \\ - 2 & 3 & - 4 \\ \end{array} \right) $$
==============================================
$$ E_{2} = \left( \begin{array}{rrr} 1 & 0 & 2 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right) $$ $$ P_{2} = \left( \begin{array}{rrr} 1 & - 1 & 2 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right) , \; \; \; Q_{2} = \left( \begin{array}{rrr} 1 & 1 & - 2 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right) , \; \; \; D_{2} = \left( \begin{array}{rrr} 1 & 0 & 0 \\ 0 & - 1 & 3 \\ 0 & 3 & - 8 \\ \end{array} \right) $$
==============================================
$$ E_{3} = \left( \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 3 \\ 0 & 0 & 1 \\ \end{array} \right) $$ $$ P_{3} = \left( \begin{array}{rrr} 1 & - 1 & - 1 \\ 0 & 1 & 3 \\ 0 & 0 & 1 \\ \end{array} \right) , \; \; \; Q_{3} = \left( \begin{array}{rrr} 1 & 1 & - 2 \\ 0 & 1 & - 3 \\ 0 & 0 & 1 \\ \end{array} \right) , \; \; \; D_{3} = \left( \begin{array}{rrr} 1 & 0 & 0 \\ 0 & - 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right) $$
==============================================
$$ P^T H P = D $$ $$\left( \begin{array}{rrr} 1 & 0 & 0 \\ - 1 & 1 & 0 \\ - 1 & 3 & 1 \\ \end{array} \right) \left( \begin{array}{rrr} 1 & 1 & - 2 \\ 1 & 0 & 1 \\ - 2 & 1 & - 4 \\ \end{array} \right) \left( \begin{array}{rrr} 1 & - 1 & - 1 \\ 0 & 1 & 3 \\ 0 & 0 & 1 \\ \end{array} \right) = \left( \begin{array}{rrr} 1 & 0 & 0 \\ 0 & - 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right) $$ $$ Q^T D Q = H $$ $$\left( \begin{array}{rrr} 1 & 0 & 0 \\ 1 & 1 & 0 \\ - 2 & - 3 & 1 \\ \end{array} \right) \left( \begin{array}{rrr} 1 & 0 & 0 \\ 0 & - 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right) \left( \begin{array}{rrr} 1 & 1 & - 2 \\ 0 & 1 & - 3 \\ 0 & 0 & 1 \\ \end{array} \right) = \left( \begin{array}{rrr} 1 & 1 & - 2 \\ 1 & 0 & 1 \\ - 2 & 1 & - 4 \\ \end{array} \right) $$