Here is the problem and its solution (link to the source if you are interested):
Two different points $A$ and $B$ are given. Find a set of such points $M$, that $\overrightarrow{MA}\cdot\overrightarrow{MB}=k^2$, where $k$ is a given non-zero number.
The solution:
Let coordinates of points be: $A(x_a, y_a), B(x_b, y_b), M(x, y)$.
Then:
$$ \begin{align} & \overrightarrow{MA}=(x_a - x, y_a - y), \overrightarrow{MB}=(x_b - x, y_b - y), \\ & \overrightarrow{MA}\cdot\overrightarrow{MB}= (x_a - x)(x_b - x)+(y_a - y)(y_b - y)=k^2 \end{align} $$
Transform the last expression, opening brackets:
$$ x_a x_b - (x_a+x_b)x + x^2 + y_a y_b - (y_a+y_b)y + y^2 = k^2 $$
At this point I ask someone to provide a more detailed explanation, please, cause I failed to understand the next steps. What a transformation has been made here?
$$ \left( x - \frac{x_a + x_b}{2} \right)^2 - \frac{(x_a - x_b)^2}{4} + \left( y - \frac{y_a + y_b}{2} \right)^2 - \frac{(y_a - y_b)^2}{4} = k^2 \\ \left( x - \frac{x_a + x_b}{2} \right)^2 + \left( y - \frac{y_a + y_b}{2} \right)^2 = k^2 + \frac{(x_a - x_b)^2}{4} + \frac{(y_a - y_b)^2}{4} $$
Thus, this is a circumference with a center at the middle of segment $AB$ and its radius. (This I also failed to understand)
$$ r = \sqrt{k^2 + \frac{|AB|^2}{4}} $$
Are you familiar with how to "complete the square?" To get from
$$ x_a x_b - (x_a+x_b)x + x^2 + y_a y_b - (y_a+y_b)y + y^2 = k^2 $$
to
$$ \left( x - \frac{x_a + x_b}{2} \right)^2 - \frac{(x_a - x_b)^2}{4} + \left( y - \frac{y_a + y_b}{2} \right)^2 - \frac{(y_a - y_b)^2}{4} = k^2 \\ $$
consider the following:
$$ \begin{align} x_ax_b-(x_a+x_b)x+x^2&=x_ax_b-(x_a+x_b)x+x^2+\frac{(x_a - x_b)^2}{4}-\frac{(x_a - x_b)^2}{4}\\ &=\frac{4x_ax_b}{4}-(x_a+x_b)x+x^2+\frac{x_a^2-2x_ax_b+x_b^2}{4}-\frac{(x_a - x_b)^2}{4}\\ &=-(x_a+x_b)x+x^2+\frac{x_a^2+2x_ax_b+x_b^2}{4}-\frac{(x_a - x_b)^2}{4}\\ &=-(x_a+x_b)x+x^2+\frac{(x_a + x_b)^2}{4}-\frac{(x_a - x_b)^2}{4}\\ &=\left( x - \frac{x_a + x_b}{2} \right)^2-\frac{(x_a - x_b)^2}{4}.\\ \end{align} $$
Similarly,
$$ \begin{align} y_ay_b-(y_a+y_b)y+y^2&=\left( y - \frac{y_a + y_b}{2} \right)^2-\frac{(y_a - y_b)^2}{4}.\\ \end{align} $$
As for the second part of your question, the standard equation for a circle with center at $P(h,k)$ and radius $r$ is $(x-h)^2+(y-k)^2=r^2$. Therefore,
$$ \left( x - \frac{x_a + x_b}{2} \right)^2 + \left( y - \frac{y_a + y_b}{2} \right)^2 = k^2 + \frac{(x_a - x_b)^2}{4} + \frac{(y_a - y_b)^2}{4} $$
is the equation for the circle with center at $$O\left(\frac{x_a+x_b}{2},\frac{y_a+y_b}{2}\right)$$ and radius
$$\begin{align} r&=\sqrt{k^2 + \frac{(x_a - x_b)^2}{4} + \frac{(y_a - y_b)^2}{4}}\\ &=\sqrt{k^2 + \frac{|AB|^2}{4}}. \end{align} $$
Note that the coordinates of $O$ are the coordinates for the midpoint of segment $AB$.
Hope this helps.