Let $A=\begin{pmatrix} 2 & 2 & 5 \\ 2 & 1 & 3 \\ 5 & 3 & \frac{17}{2} \end{pmatrix} \in M_3(\mathbb{R})$.
I want to find an invertible matrix $C$ such that $C^TAC=\begin{pmatrix} 1 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & 0 \end{pmatrix}$, since there are three eigenvalues which are postive, negative and zero.
I tried to use completing the square with the bilinear form $s(v,v)=\langle v,Av \rangle$:
$s(v,v)= \langle v,Av \rangle=(v_1,v_2,v_3)\begin{pmatrix} 2 & 2 & 5 \\ 2 & 1 & 3 \\ 5 & 3 & \frac{17}{2}\end{pmatrix}\begin{pmatrix} v_1\\v_2\\v_3 \end{pmatrix}=2v_1^2+4v_1v_2+10v_1v_3+v_2^2+6v_2v_3+\frac{17}{2}v_3^2$
Now I used completing the square with respect to $v_1$:
$2(v_1^2+2v_1v_2+5v_1v_3)+v_2^2+6v_2v_3+\frac{17}{2}v_3^2$
$=2(v_1^2+v_1(2v_2+5v_3))+v_2^2+6v_2v_3+\frac{17}{2}v_3^2$
$=2(v_1^2+v_1(2v_2+5v_3)+\frac{1}{2}(2v_2+5v_3)^2-\frac{1}{2}(2v_2+5v_3)^2)+v_2^2+6v_2v_3+\frac{17}{2}v_3^2$
$=2(v_1+(2v_2+5v_3))^2-\frac{1}{2}(2v_2+5v_3)^2+v_2^2+6v_2v_3+\frac{17}{2}v_3^2$
$=2(v_1+2v_2+5v_3)^2-\frac{1}{2}(2v_2+5v_3)^2+(v_2+3v_3)^2-\frac{1}{2}v_3^2$
Now there is one summand too much so I can't set a matrix $C$ which is the inverse of the matrix with the entries in the braces.
I think I made a mistake with the completing the square but I don't see where.
$$2x^2+4xy+10xz+y^2+6yz+\frac{17}2z^2=2\left(x+y+\frac52z\right)^2-y^2-4yz-4z^2=$$
$$=2\left(x+y+\frac52z\right)^2-\left(y+2z\right)^2$$
and there you go: one positive, one negative, one zero...