This question is related with machine learning. $ \\ $But I assumed linear algebra people will be able to help.
The most basic classification form
$$y(x) = w^Tx + w_0$$
$w \ $ is called weight vector.
These equations give me some trouble to understand (not good with linear algebra).
Classification linear regression :
- $x = x_{\bot} + r \frac{w}{||w||}$
- $r = \frac{y(x)}{||w||}$
Not sure which part of linear algebra or vector space should I look up to understand this picture.
$\frac{y(x)}{||w||}$ , $\frac{-w_0}{||w||}$ : where these guys come from and represent?
And final equation
Can anyone help me to understand this equation simply?
Okay so the figure explains the equations $y(x) = w^T x + w_0$, for a 2D-case, namely when the vectors $w$ and $x$ are 2 dimensional. Keep in mind that vector $w$ is the normal (or directional) vector to line $w^T x$ and $w_0$ is just an intercept, i.e. it shifts the line $w^T x$ that passes thru the origin (black line) to $y(x) = w^T x + w_0$.
-The notation $x_{\perp}$ got me a bit confused, but I quite understood it by doing the following: \begin{equation} \begin{split} y(x) &= w^Tx + w_0 \\&= w^T(x_{\perp} + r\frac{w}{\Vert w \Vert}) + w_0 \\&= w^T x_{\perp} + r\Vert w \Vert + w_0 \\&= w^T x_{\perp} + \frac{y(x)}{\Vert w \Vert}\Vert w \Vert + w_0 \\&= w^T x_{\perp} + y(x) + w_0 \end{split} \end{equation} which means $w^T x_{\perp} + w_0 = 0$. Therefore, the point $x_{\perp}$ is such that $y(x_{\perp}) = 0$, therefore the red line. P.S: It got me confused because usually $x_{\perp}$ means a vector that nulls $x$, i.e. $x^T x_{\perp} = 0$, but this is not what you mean in the figure.
-First answer to: where did $\frac{y(x)}{\Vert w \Vert}$ come from ? Well, this is the distance between any point $x$ and the line $y(x_{\perp})$ (the red line). Using the following relation which gives the distance between point $x$ and line $y(x_{\perp})$, we have \begin{equation} \text{dist}(y(x_{\perp}),x) = \frac{\vert y(x) \vert }{\Vert w \Vert} \end{equation} where we can remove the $\vert \vert$ since $y(x)$ is positive in the figure above. [Reference, see https://en.wikipedia.org/wiki/Distance_from_a_point_to_a_line].
-Second answer to where did $-\frac{w_0}{\Vert w \Vert}$ come from: It is the distance between two parallel lines: The red line $y(x_{\perp})$ and the parallel black line say $z(x) = w^Tx$ since they have same directional vector $w$ and the black line passes thru the origin (so no intercept). Hence \begin{equation} \text{dist}(y(x_{\perp}),z(x)) = \frac{\vert w_0 - 0 \vert }{\Vert w \Vert} \end{equation} where the negative sign in the figure appears because it is towards the negative side. [Reference, see https://en.wikipedia.org/wiki/Distance_between_two_straight_lines].