Understanding Linear classifiers

Question

I'm studying about linear classifiers. We learned that points above a line satisfy $ax + b\ge 0$ and points that below the line satisfy $ax+b < 0$.

Why is it so?

More generally $\boldsymbol{w}^T\boldsymbol{x}$ instead of $ax$ where $\boldsymbol{w} \in \mathbb{R}^d$.

Answer 1

To describe a plane in the $n$ dimensional space we can use the following formula

$$\boldsymbol{w}^T\left[\boldsymbol{x}-\boldsymbol{a} \right] = 0.$$

This is equivalent to say that normal vector $\boldsymbol{w}^T$ is perpendicular to any vector in the plane $\left[\boldsymbol{x}-\boldsymbol{a} \right]$, in which $\boldsymbol{a}$ is an arbitrary point in the plane. Note, that $\boldsymbol{a}^T\boldsymbol{b}$ is the matrix notation for the dot product. The vectors $\boldsymbol{a}$ and $\boldsymbol{b}$ are perpendicular to each other if the dot product $\boldsymbol{a}^T\boldsymbol{b}=0$.

If you expand this formula you obtain:

$$\boldsymbol{w}^T\boldsymbol{x}-\boldsymbol{w}^T\boldsymbol{a} = 0,$$

in which $-\boldsymbol{w}^T\boldsymbol{a}$ is just a constant scalar which was called $b$ in your expression.

Answer 2

In 1D, assuming $a>0$,

$$ax+b\ge0\iff x\ge{-\frac ba}.$$

This describes a half-line on $\mathbb R$.

Now in $d$ dimensions, $\boldsymbol{w}^T\boldsymbol{x}$ denotes the orthogonal projection of the vector $\boldsymbol{x}$ onto an axis from the origin and parallel to $\boldsymbol{w}$ (property of the scalar product). This reduces the locus to 1D. Now we have all the points that project to a halfline, and these form a half space.