Evaluate Cluster Models?

NatalySimth

What potential problem will we encounter if we only use Avg. within centroid distance as the main criterion for evaluating clustering models?

Telcontar120

The question as posed is somewhat vague, I think. Remember that with unsupervised ML algorithms, there is no "correct" answer a priori that the model can use to evaluate the quality of the output against some known end state. So usually the most important starting point is to ask what is the purpose of the unsupervised ML approach and how will you measure whether you have fulfilled that purpose? That may lead you to whether a particular metric is going to be a good metric for your purposes. Avg within centroid distances will tell you how "close" in multidimensional space your observations are within each cluster, but without knowing why you are doing clustering in the first place there is no way of knowing whether this is a good metric, or how this metric should be weighed against other tradeoff factors such as adding an additional cluster (i.e., having more clusters will typically tend to reduce the within centroid distances, but you may not want a large number of clusters).