This is just an eg. to understand how to solve it.
There is a quadratic form as below we need to bring to canonical form using Lagrange's method and find the coordinate transformation.
Then find the positive & negative indexes, rank of the quadratic form. Finnally, to investigate sign-definiteness of the quadratic form by two ways: according to it's canonical form and the criterion of Sylvester: $4x_{1}^2+2x_{2}^2+10x_{3}^2-4x_{1}x_{2}-12x_{1}x_{3}-8x_{2}x_{3}$
Any clarification on how to do it are very welcomed.
Lagrange's method, as such, would give $$ (2x-y-3z)^2 +(y-7z)^2 - 48 z^2 $$ for two positive and one negative eigenvalues.
$$ Q^T D Q = H $$ $$\left( \begin{array}{rrr} 1 & 0 & 0 \\ - \frac{ 1 }{ 2 } & 1 & 0 \\ - \frac{ 3 }{ 2 } & - 7 & 1 \\ \end{array} \right) \left( \begin{array}{rrr} 4 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & - 48 \\ \end{array} \right) \left( \begin{array}{rrr} 1 & - \frac{ 1 }{ 2 } & - \frac{ 3 }{ 2 } \\ 0 & 1 & - 7 \\ 0 & 0 & 1 \\ \end{array} \right) = \left( \begin{array}{rrr} 4 & - 2 & - 6 \\ - 2 & 2 & - 4 \\ - 6 & - 4 & 10 \\ \end{array} \right) $$
There is a method that is the reverse of Lagrange, and done carefully gives both directions,
Algorithm discussed at http://math.stackexchange.com/questions/1388421/reference-for-linear-algebra-books-that-teach-reverse-hermite-method-for-symmetr
https://en.wikipedia.org/wiki/Sylvester%27s_law_of_inertia
$$ H = \left( \begin{array}{rrr} 4 & - 2 & - 6 \\ - 2 & 2 & - 4 \\ - 6 & - 4 & 10 \\ \end{array} \right) $$ $$ D_0 = H $$ $$ E_j^T D_{j-1} E_j = D_j $$ $$ P_{j-1} E_j = P_j $$ $$ E_j^{-1} Q_{j-1} = Q_j $$ $$ P_j Q_j = Q_j P_j = I $$ $$ P_j^T H P_j = D_j $$ $$ Q_j^T D_j Q_j = H $$
$$ H = \left( \begin{array}{rrr} 4 & - 2 & - 6 \\ - 2 & 2 & - 4 \\ - 6 & - 4 & 10 \\ \end{array} \right) $$
==============================================
$$ E_{1} = \left( \begin{array}{rrr} 1 & \frac{ 1 }{ 2 } & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right) $$ $$ P_{1} = \left( \begin{array}{rrr} 1 & \frac{ 1 }{ 2 } & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right) , \; \; \; Q_{1} = \left( \begin{array}{rrr} 1 & - \frac{ 1 }{ 2 } & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right) , \; \; \; D_{1} = \left( \begin{array}{rrr} 4 & 0 & - 6 \\ 0 & 1 & - 7 \\ - 6 & - 7 & 10 \\ \end{array} \right) $$
==============================================
$$ E_{2} = \left( \begin{array}{rrr} 1 & 0 & \frac{ 3 }{ 2 } \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right) $$ $$ P_{2} = \left( \begin{array}{rrr} 1 & \frac{ 1 }{ 2 } & \frac{ 3 }{ 2 } \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right) , \; \; \; Q_{2} = \left( \begin{array}{rrr} 1 & - \frac{ 1 }{ 2 } & - \frac{ 3 }{ 2 } \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right) , \; \; \; D_{2} = \left( \begin{array}{rrr} 4 & 0 & 0 \\ 0 & 1 & - 7 \\ 0 & - 7 & 1 \\ \end{array} \right) $$
==============================================
$$ E_{3} = \left( \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 7 \\ 0 & 0 & 1 \\ \end{array} \right) $$ $$ P_{3} = \left( \begin{array}{rrr} 1 & \frac{ 1 }{ 2 } & 5 \\ 0 & 1 & 7 \\ 0 & 0 & 1 \\ \end{array} \right) , \; \; \; Q_{3} = \left( \begin{array}{rrr} 1 & - \frac{ 1 }{ 2 } & - \frac{ 3 }{ 2 } \\ 0 & 1 & - 7 \\ 0 & 0 & 1 \\ \end{array} \right) , \; \; \; D_{3} = \left( \begin{array}{rrr} 4 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & - 48 \\ \end{array} \right) $$
==============================================
$$ P^T H P = D $$ $$\left( \begin{array}{rrr} 1 & 0 & 0 \\ \frac{ 1 }{ 2 } & 1 & 0 \\ 5 & 7 & 1 \\ \end{array} \right) \left( \begin{array}{rrr} 4 & - 2 & - 6 \\ - 2 & 2 & - 4 \\ - 6 & - 4 & 10 \\ \end{array} \right) \left( \begin{array}{rrr} 1 & \frac{ 1 }{ 2 } & 5 \\ 0 & 1 & 7 \\ 0 & 0 & 1 \\ \end{array} \right) = \left( \begin{array}{rrr} 4 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & - 48 \\ \end{array} \right) $$ $$ Q^T D Q = H $$ $$\left( \begin{array}{rrr} 1 & 0 & 0 \\ - \frac{ 1 }{ 2 } & 1 & 0 \\ - \frac{ 3 }{ 2 } & - 7 & 1 \\ \end{array} \right) \left( \begin{array}{rrr} 4 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & - 48 \\ \end{array} \right) \left( \begin{array}{rrr} 1 & - \frac{ 1 }{ 2 } & - \frac{ 3 }{ 2 } \\ 0 & 1 & - 7 \\ 0 & 0 & 1 \\ \end{array} \right) = \left( \begin{array}{rrr} 4 & - 2 & - 6 \\ - 2 & 2 & - 4 \\ - 6 & - 4 & 10 \\ \end{array} \right) $$