I'm not able to understand the concepts behind the following problem that appeared on my homework.
Consider the line defined by ${w}^\mathsf{T}x + b = 0$ and two points $x_1$ and $x_2$ that lie on the line:
- Calculate ${w}^\mathsf{T}\left(x_1 - x_2\right)$
I used substitution(substituting x1,x2 in line equation) to derive that the answer is $0$.
- This tells us that ${w}$ is [orthogonal/collinear] to the line.
From trial and error, I figured out the answer is orthogonal (it was a multiple choice question).
What topics should I revisit/study to visualize and solve such questions?
if $w \neq 0$ such that $w^Tx+b = 0$, and since $x_1$ and $x_2$ lies on the line, then they must satisfy the equation.
$$w^Tx_1 + b = 0$$ $$w^Tx_2+b=0$$
Subtracting the equations, we have
$$w^T(x_1-x_2)=0$$
We can write the above as an inner product.
$$\langle w, x_1-x_2 \rangle=0$$
$$\|w\| \|x_1-x_2\| \cos \theta =0$$
If $w$ and $x_1-x_2$ are non-zero, then $\cos \theta = 0$, that is $\theta$ must be an odd multiple of $90^\circ$.
Hence $w$ and $x_1-x_2$ is perpendicular.
$x_1-x_2$ is the direction of the line, hence $w$ must be perpendicular to the line.
Potential relevant topic: inner product, multivariate calculus, linear algebra, vector.