Let $E$ be the vectorspace $\mathbb{R}^2$. A general quadratic form on $E$ looks like:
$q(x,y)=ax^2+bx y+cy^2$, where $(x,y)$ is a vector in $E$.
Interesting examples are
$q(x,y)=x^2+y^2$
$q(x,y)=x^2-y^2$
what is the associated bilinear form for $q$?
Now to put this into context, the book referred is Hyperbolic Geometry by Birger Iverson.
On page $1$ it says
$$ \left< x,y \right>=\frac{Q(x+y)-Q(x)-Q(y)}{2}$$
where $x,y \in E$ is bilinear in $x$ and $y$.
The variables $x$ and $y$ are used confusingly in your question: in the first part they denote coordinates of the same vector, while in the second part they denote two different vectors. For this reason I'll try to avoid using them at all, and call my vectors $v=(v_1,v_2)$ and $w=(w_1,w_2)$.
The bilinear form associated with $q(v)=av_1^2+bv_1v_2+cv_2^2$ would be
$$\langle v,w\rangle=av_1w_1 + b(v_1w_2+v_2w_1)/2 + cv_2w_2$$
In order to read that from the equation you quoted, observe
\begin{align*} Q(v+w)&=a(v_1+w_1)^2+b(v_1+w_1)(v_2+w_2)+c(v_2+w_2)^2 \\&= a(v_1^2+2v_1w_1+w_1^2)+b(v_1v_2+v_1w_2+v_2w_1+v_2w_2) +c(v_2^2+2v_2w_2+w_2^2) \\&=Q(v)+Q(w)+2av_1w_1+b(v_1w_2+v_2w_1)+2cv_2w_2 \end{align*}
so
$$\frac{Q(v+w)-Q(v)-Q(w)}{2}= av_1w_1 + b(v_1w_2+v_2w_1)/2 + cv_2w_2$$
as claimed.