Tight Clustering

[mepcotterell]

We need to implement and test Tight Clustering [1].

References

[1] Tseng, George C., and Wing H. Wong. 2005. “Tight Clustering: A Resampling-Based Approach for Identifying Stable and Tight Patterns in Data.” Biometrics 61 (1). Blackwell Publishing: 10–16. doi:10.1111/j.0006-341X.2005.031032.x.

[mepcotterell]

Starter code for tight clustering introduced in commit 9652985. Currently, it is the same as the KMeansClustering class.

[mepcotterell]

Started the actual implementation in commit 3390b9b.

[mepcotterell]

Continued working on an implementation in commit 180b552. One of the biggest improvements is the use of a scalation.linalgebra.SparseMatrixD when computing the average comembership matrix of multiple random clusterings.

[mepcotterell]

I'm currently waiting on the Sapelo cluster to be restored. While I can test the code on smaller datasets, I need sapelo in order to accommodate the comembership matrices for our clustering problem. For small values of k (# number of clusters), the comembership matrices are quite dense (as expected). As k increases, they become more sparse. Still, we're dealing with matrices that are > 91k-by-91k (in the number of elements).

[mepcotterell]

New starter code for the TightClusterer class has been added to the scalation_1.3 portion of the code base as of commit 66086c0.