An example problem from Serge Lang's Calculus of Several Variables (pg. 30-31):
Example 3. Find the equation of the plane passing through the three points
$$ P_1 = (1,2, -1), P_2 = (-1, 1, 4), P_3 = (1,3,-2) $$
We visualize schematically the three points as follows:
[picture of triangle omitted]
Then we find a vector $N$ perpendicular to $\overrightarrow{P_1 P_2}$ and $\overrightarrow{P_1 P_3}$, or in other words, perpendicular to $P_2 - P_1$ and $P_3 - P_1$. We have
\begin{align} P_2 - P_1 &= (-2, -1, +5) \\ P_3 - P_1 &= (0, 1, -1) \\ \end{align}
Let $N = (a, b, c)$ . We must solve
\begin{align} -2a - b + 5c &= 0, \\ b - c &= 0. \end{align}
We take $b = c = 1$ and solve for $a$, getting $a = 2$. Then
$$ N = (2,1,1) $$
satisfies our requirements. The plane perpendicular to $N$, passing through $P_1$ is the desired plane. Its equation is therefore
$$ 2x + y + z = 2 + 2 - 1 = 3 $$
Question: Why were the two equations set to $0$ in
\begin{align} -2a - b + 5c &= 0, \\ b - c &= 0? \end{align}
What is going on here?
Because the equations $N \cdot (P_2-P_1) = 0$ and $N \cdot (P_3-P_1) = 0$ define the plane. The normal is orthogonal to any vector that lies in the plane, i.e. the dot product of the normal and any vector in the plane is zero. Since $N = (a,b,c)$: $$(a,b,c) \cdot (-2,-1,5) = -2a-b+5c = 0$$ $$(a,b,c) \cdot (0,1,-1) = b-c = 0$$