Gaussian kernel's parameter

208 Views Asked by At

I learned from machine learning course that larger $\sigma^2$, leads to higher bias.

enter image description here

In this paper also mentioned larger bandwidth cause uniform kernel matrix. Which leads to high bias.

Then this note shows: enter image description here enter image description here

Based on my understanding: the picture on the left is more bias which means larger $\sigma^2$, but it has smaller $\gamma = \sigma^2$. This is first contradiction I encountered.

Then I tried to think what happen when $\sigma^2$ get larger. As $\sigma^2$ approaches $\infty$ then $exp(\frac{-\|u-v\|^2}{\sigma^2})$ approaches $\infty$, if we consider $\|u-v\|^2$ is positive constant. While the paper I mentioned earlier wrote

a large σ value will cause the kernel matrix to be a uniform matrix with only entries 1.

This is the second contradiction I cannot understand. Why the kernel entries are 1?

How should I analyze the bias in this model? Could anyone help please to resolve this?

Thanks in advance.