cluster validation and determining number of clusters

527 Views Asked by At

I want to find number of cluster in the real world data set. So, I validate the spectral clustering by using some indexes as shown in figures below? But as you seen in figures the results are very different from one index to other. How to determine the good number of clusters in the spectral clustering method? If u don't have complete answer give me any suggestion or idea that may help me.

Thanks
enter image description here enter image description here enter image description here enter image description here enter image description here enter image description here

1

There are 1 best solutions below

10
On

I believe there exists no best method to assess clustering quality, independently of the data. You can check this answer for many other ways to assess such quality, in the case of k-means. Thiscan be good starting point: the relationship between spectral clustering and a suitable generalization of k-means is contained here .An important discrimination in quality assessment is between internal and external methods: internal ones check the dependence of an objective function on the number of clusters (for example, this is what happens with the "elbow" method). External criteria refer to an optimal "choice", which has to be known a priori, and try to compare the optimum with the computed clustering. The optimal choice has to be inferred looking at the data, knowing the problem in detail, or by other methods. In your specific case, different methods seem to give quite different answers. I would try now to visualize the clusters, and to find and select the choice which gives the most reasonable classification of the data themselves.