Understanding the use of Radial Basis Function in Linear Regression

598 Views Asked by At

I am attempting to understand the use of Radial Basis Functions (RBFs) as used in linear regression.

Building the problem:

RBFs can be used as a means of separating data which is not linearly separable (see example scatter-plot by link containing different below)

Non-linearly separable Scatter plot

Positive (green) and negative (red) data-points are clustered together but no straight line can be found to separate them from each-other.

We are seeking some function $f(x) = w\phi(x) + c$ which will plot a 'line' between the points. Note:

  • $w$ represents a matrix of weights
  • $\phi(x) = \exp(−(x−c)^⊤(x−c)/h^2)$ (Note: Our Radial Basis function is Gaussian).
  • $c$ is our 'y' intercept
  • $h$ relates to how quickly $\phi(x)$ drops off towards zero

My current "understanding":

We can use a linear combination of RBFs (a seperate RBF for each cluster, 3 in this case) to find a 'line' ( $f(x)$ ) which will separate positive data-points from negative data-points.

The function will appear as follows:

$f(x) = w_1\phi_1(x) + w_2\phi_2(x) + w_3\phi_3(x)$

So we choose values for centers $c_1, c_2, c_3$ to offset the respective RBFs $\phi_1(x), \phi_2(x), \phi_3(x)$ according to the distance the related scatter plot is from the origin of the 2-d plane ($x_1$,$x_2$).

Appropriate values are then chosen for the $w_1, w_2, w_3$ matrix such that when multiplied by the respective $\phi$ the hope being that any new (previously unseen values) will be determined to be positive or negative depending on what side of the 'line' defined by $f(x)$ they reside on.

My Questions:

  1. Am I correct, incorrect, partially correct in my current understanding?
  2. If correct are there things you feel I am leaving out?
  3. If incorrect, why?
  4. If partially correct, could you advise what is right or wrong?