Rosenblatt Perceptron theory question.

43 Views Asked by At

I am currently going over Rosenblatt Perceptron's and its generalizations. I understand that, normally, the activation function is given by

$$ \varphi (a) = \begin{cases} 1 , \quad \text{ if } a \geqslant 0 \\ -1, \quad \text{otherwise}.\end{cases}$$

Therefore, the Rosenblatt Perceptron can only define a linear decision boundary, which is given by $$ w^\top \tilde x = 0,$$ where $w^\top = (w_1,\dots,w_D,w_0)^\top$ and $\tilde x = (x_1,\dots,x_D,1).$

Now, when expanding my study to some generalizations of the Rosenblatt Perceptron, I came across the logistic activation function, which is given by (as far as my understanding goes)

$$ \sigma (a) = \frac{1}{1+e^{-a}}.$$

It is of trivial observation to check that $\sigma(a) \in \, ]0,1[,$ which is a difference when compared with the activation function I described above by $\varphi(\cdot).$ Normally we set a threshold of $0.5$ and, in binary classification problems, we assume that if $\sigma(w^\top \tilde x) \geqslant 0.5,$ we classify the observation $x$ in the class corresponding to $t=1$ and otherwise we classify $x$ in the class corresponding to $t=0.$

I am now trying to understand what decision boundary is defined by this logistic activation function. With the threshold fixed at $0.5,$ my logic tells me that this boundary decision is given by

$$ \frac{1}{1+e^{-w^\top \tilde x}} = 0.5 \Leftrightarrow 1 + e^{-w^\top \tilde x} = 2 \Leftrightarrow e^{-w^\top \tilde x} = 1 \Leftrightarrow w^\top \tilde x =0. $$

This leads me to the conclusion that the decision boundary is the same using the activation function $\varphi$ or using the logistic activation function I denoted by $\sigma.$

Is my thinking correct?

Thanks for any help in advance.