Abstract

Clustering is a fundamental problem in many scientific applications. Standard methods such as k-means, Gaussian mixture models, and hierarchical clustering, however, are beset by local minima, which are sometimes drastically suboptimal. Recently introduced convex relaxations of k-means and hierarchical clustering shrink cluster centroids toward one another and ensure a unique global minimizer. In this work, we present two splitting methods for solving the convex clustering problem. The first is an instance of the alternating direction method of multipliers (ADMM); the second is an instance of the alternating minimization algorithm (AMA). In contrast to previously considered algorithms, our ADMM and AMA formulations provide simple and unified frameworks for solving the convex clustering problem under the previously studied norms and open the door to potentially novel norms. We demonstrate the performance of our algorithm on both simulated and real data examples. While the differences between the two algorithms appear to be minor on the surface, complexity analysis and numerical experiments show AMA to be significantly more efficient. This article has supplementary materials available online.

Highlights

  • In recent years convex relaxations of many fundamental, yet combinatorially hard, optimization problems in engineering, applied mathematics, and statistics have been introduced (Tropp, 2006)

  • Given the 1 norm and unit weights wij, the objective function separates, and they solve the convex clustering problem by the exact path following method designed for the fused lasso (Hoefling, 2010)

  • We review the alternating direction method of multipliers (ADMM) and alternating minimization algorithm (AMA) algorithms and adapt them to solve the convex clustering problem

Read more

Summary

Introduction

In recent years convex relaxations of many fundamental, yet combinatorially hard, optimization problems in engineering, applied mathematics, and statistics have been introduced (Tropp, 2006). In this case, the objective function Fγ(U) separates over the connected components of the underlying graph. Because the objective function Fγ(U) in equation (1.1) is strictly convex and coercive, it possesses a unique minimum point for each value of γ. 0.3 x q qqqqqqqqqqqqqqqq qqq Figure 2: Cluster path assignment: The simulated example shows five well separated clusters and the assigned clustering from applying the convex clustering algorithm using an 2-norm. The classical greedy algorithm for solving k-means clustering often gets trapped in suboptimal local minima (Forgy, 1965; Lloyd, 1982; MacQueen, 1967).

Contributions
Related Work
Notation
Organization
Properties of the solution path
Algorithms to Compute the Clustering Path
Reformulation of Convex Clustering
Proximal Map
ADMM updates
AMA updates
Stopping Criterion for AMA
Convergence
Acceleration
Computational Complexity
Choosing weights
Making cluster assignments
Numerical Experiments
Qualitative Comparisons
Two Half Moons
Fisher’s Iris Data
Senate Voting
Dentition of mammals
Timing Comparisons
Conclusion & Future Work
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call