I am attempting to understand the use of Radial Basis Functions (RBFs) as used in linear regression.
Building the problem:
RBFs can be used as a means of separating data which is not linearly separable (see example scatter-plot by link containing different below)
Non-linearly separable Scatter plot
Positive (green) and negative (red) data-points are clustered together but no straight line can be found to separate them from each-other.
We are seeking some function $f(x) = w\phi(x) + c$ which will plot a 'line' between the points. Note:
- $w$ represents a matrix of weights
- $\phi(x) = \exp(−(x−c)^⊤(x−c)/h^2)$ (Note: Our Radial Basis function is Gaussian).
- $c$ is our 'y' intercept
- $h$ relates to how quickly $\phi(x)$ drops off towards zero
My current "understanding":
We can use a linear combination of RBFs (a seperate RBF for each cluster, 3 in this case) to find a 'line' ( $f(x)$ ) which will separate positive data-points from negative data-points.
The function will appear as follows:
$f(x) = w_1\phi_1(x) + w_2\phi_2(x) + w_3\phi_3(x)$
So we choose values for centers $c_1, c_2, c_3$ to offset the respective RBFs $\phi_1(x), \phi_2(x), \phi_3(x)$ according to the distance the related scatter plot is from the origin of the 2-d plane ($x_1$,$x_2$).
Appropriate values are then chosen for the $w_1, w_2, w_3$ matrix such that when multiplied by the respective $\phi$ the hope being that any new (previously unseen values) will be determined to be positive or negative depending on what side of the 'line' defined by $f(x)$ they reside on.
My Questions:
- Am I correct, incorrect, partially correct in my current understanding?
- If correct are there things you feel I am leaving out?
- If incorrect, why?
- If partially correct, could you advise what is right or wrong?