Perceptron training in Python

Question

I am trying to understand how the weights update in a very simple perceptron training model, given here: The code that I am following

Training points, labels and the answer that I am trying to reach

w = np.array([5, 2, -3]) ## real weights 
x = [[5,2], [-2,8], [-1,-3], [-3,-2], [3,-7], [3,2], [1,3], [0,0], [0,1], [4,4]] ## x
y = np.array([1, 1, -1, -1, -1, 1, 1, -1, -1, 1]) ## y

Plot of points and the real weight : Plot

My training model :

## initial weights
w_t = np.array([1,1,1])

## xtrain
x_t = x
for t in x_t:
    t.append(1)
x_t = np.array(x_t)

while(i < 20):
    print(w_t[0],w_t[1],w_t[2])    
    y_pred = perc(x_t, w_t)
    if(check(y, y_pred)==False):
        w_t = w_t + np.matmul(y.T,x_t)
        i+=1
        
    else:
        break

The check function which checks if all the points are seperated :

def check(y, y_pred): ## function to check if all predictions are equal or not
    for i in range(len(y)):
        if(y[i]!=y_pred[i]):
            return False
    
    return True

The perc model that predicts the labels (-1 or 1) :

def perc(x_t, w_t): ## function to predict
    yp = np.matmul(x_t, w_t.T).T
    return yp/abs(yp)
    

The weights seems to be diverging, and the bias term is not updating in each iteration, here is what i got when i trained it for 20 iterations :

1 1 1
13 31 1
25 61 1
37 91 1
49 121 1
61 151 1
73 181 1
85 211 1
97 241 1
109 271 1
121 301 1

Answer 1

Follows a MATHEMATICA script which implements the cited pseudo code. You can compare it with your python code to correct it.

x = {{5, 2}, {-2, 8}, {-1, -3}, {-3, -2}, {3, -7}, {3, 2}, {1, 3}, {0, 0}, {0, 1}, {4, 4}};
y = {1, 1, -1, -1, -1, 1, 1, -1, -1, 1};
Clear[percept]
percept[w_List, x_List] := Sign[w.x]

w = {1, 1, 1};
kmax = 20;

For[k = 0, k <= kmax, k++,
 For[j = 1, j <= Length[x], j++,
  xj = Join[x[[j]], {1}];
  yc = percept[w, xj];
  If[yc != y[[j]], w += y[[j]] xj]
  ]
 ]

gr1 = Show[Table[If[y[[k]] == 1, 
 Graphics[{Red, PointSize[0.03], Point[x[[k]]]}], 
 Graphics[{Blue, PointSize[0.03], Point[x[[k]]]}]], {k, 1, Length[x]}]];
z = w.{X, Y, 1};
gr2 = ContourPlot[z == 0, {X, -8, 8}, {Y, -8, 8}, ContourStyle -> Black];
Show[gr1, gr2, AspectRatio -> 1]

NOTE

In python

import numpy as np

x = [[5, 2], [-2, 8], [-1,-3], [-3,-2], [3,-7], [3, 2], [1, 3], [0, 0], [0, 1], [4, 4]]
y = [1, 1, -1, -1, -1, 1, 1, -1, -1, 1]
n = len(x)
w = [1, 1, 1]
m = len(w)
kmax = 20

def percept(w, x):
    s = 0
    for i in range(m):
        s += x[i]*w[i]
    return np.sign(s)

for k in range(kmax):    
    for j in range(n):        
        xj = []
        for x0 in x[j]:
            xj.append(x0)
        xj.append(1)
        yc = percept(w, xj)
        if yc != y[j]:
            for i in range(m):
                w[i] += y[j]*xj[i]
            
print (w)