Linear Separator

Question

First of all, sorry for my bad english. How can be an artificial neuron with 'n' inputs and 'n' weights a linear separator? I saw a lot of courses when they showed me those artificial neuron ( first classes of those courses ) with 'N' inputs and weights and from the other hand they showed me a Descartes coordinate system with a line witch is the separator line of that neuron's output. In the coordinate system I saw points from sector A and B , separated by a line. What does that neuron, and its inputs and weights have to do with those points in the coordinate system ? The coordinate system has two axis, x and y with corresponding points like ...-2,-1,0,1,2... meanwhile that neuron just has 'N' input witch can be either 0 or 1 multiplied by the weights and sumed up. Whats the point on this ? Thank you !

Answer 1

I think you are just getting confused by the 'linear' part.The line just so happens to be a linear equation.Every linear equation has the following format $ax + b=y$ .This corresponds to the neuron - $x$ is your input and $a$ is the weight associated with it and $b$ is the bias.

In the neural network you have a bunch of these :

$\sum_{i=0}^{n} w_{i}x_{i} + b_{i}$

I deliberately used summation so that you can see that it still has the same format.

Now let's look at the graph if we have for example a weight of 1

w=1

Our function will draw a line straight trough the diagonal and continue to where x has been defined (if your input is $[-1;1]$ the solution space of your equation $1x=y$ will be a square).

Now let's see how this changes when we change the weight :

w=0.3

It just rotated the line segment.So in essence you can think of it like this- weights are rotating the line that separates the output, biases shift the line segment, and how you define your input determines how the solution space expands.

You were a little confused because they showed you a binary classification with only one input for simplicity.The fact is that you can't visualize more than 2 dimensional input, but the underlying principle stays the same for $n$ dimensions.Just sum up every weight*input + bias and you will get a scalar that corresponds to the output.

Ultimately what the training of the network does is to find that unique line segment, by rotating and shifting it to produce two sepate areas where the maximum amound of examples fall in with minumum amount of error.