Intuitive Way of Thinking about $\boldsymbol{w}^T\boldsymbol{x} + b = 0$.

Question

I can understand $y = ax + b$ as $x$ represents how much you go up and to the right per increment of $b$, $b$ represents the $y$-intercept (due to solving for $y$ given $x = 0$ yielding $b$). However, I cannot understand the $\boldsymbol{w}^T\boldsymbol{x} + b = 0$. The math checks out, given that $x$ is $(x, y)$ and $w$ stores the coefficients with some modifications. Can someone offer an intuitive explanation of how a hyperplane can be described by it's perpendicular vector, and why multiplying x by a perpendicular vector will yield a point on a line that is perpendicular to the vector it was inner-product-ed with?

Answer 1

Let us refer to the normal vector as $\boldsymbol{w}$. Additionally, we define $\boldsymbol{a}$ to be an arbitrary constant point in the hyperplane and $\boldsymbol{x}$ to be an additional arbitrary variable point in the plane.

If we set ut the vector from $\boldsymbol{a}$ to $\boldsymbol{x}$ (both points in the plane) by $\boldsymbol{x}-\boldsymbol{a}$, then we know that this vector (or any Vector inside the hyperplane) is orthogonal to the normal vector. Orthogonality can be checked by the dot product. The dot product of orthogonal vectors is equal to zero.

Hence we can write (by using the dot product in vector notation)

$$\boldsymbol{w}^T\left[\boldsymbol{x}- \boldsymbol{a}\right]=0 \qquad \text{or} \qquad \boldsymbol{w}^T\boldsymbol{x}-\boldsymbol{w}^T\boldsymbol{a}=0 .$$

In the last expression substituting $b=-\boldsymbol{w}^T\boldsymbol{a}$ results in your representation.