Standard K-means Algorithm Research Articles

Localized density-based K-means UAV communication coverage algorithm for UV collaboration

In response to the problem of UAV-assisted communication, utilizing the characteristics of wireless ultraviolet with strong anti-jamming performance and good confidentiality, providing emergency communication services by covering ground users with UAVs as aerial base stations is crucial. A K-means clustering coverage algorithm based on the local density is proposed, which increases the connectivity term of UAVs on the basis of minimizing the squared error; secondly, the definition formula of the local density is proposed, and a heuristic decision function is used to determine the selection of the initial cluster center. The simulation results show that the initial coverage of the initial cluster center selected by the heuristic function is increased by 31.6%. Compared to the standard K-means algorithm and FSFDP algorithm, the coverage rate of the proposed algorithm in this paper is improved by 7.2% and 4.3%, respectively.

Fast but approximate homomorphic k-means based on masking technique

Nowadays, computing on encrypted data seems to be more practical than a few years ago, thanks to the emergence of new Homomorphic Encryption schemes. In this paper, an algorithm based on Homomorphic Encryption for Arithmetic of Approximate Numbers (Cheon et al., in: Takagi, Peyrin (eds) Advances in cryptology—ASIACRYPT 2017, Springer, Cham, pp 409–437, 2017) (HEAAN, or also CKKS) scheme, that is able to perform a secure k-means algorithm which processes encrypted data, has been studied and presented. The performance of the classifier running on encrypted data has been evaluated using a standard k-means algorithm that works on plain data as a supervised structure, since the results are obtained by approximated computations. The main point of this paper is to take existent theoretical techniques (for example approximations of text {sgn}(x)), to use them and to observe if they are valid in practical applications. The output of the algorithm is a set of k encrypted masks that can be applied to the original dataset in order to obtain different clusters. The setting is a standard client–server one. The workload is heavily server-centric, as the client only has to execute a light masking algorithm at the end of each iteration, which, excluding the decryption, is faster than a plain k-means iteration; the main disadvantage concerns the accuracy of the results. Experiments show that the algorithm can be executed fairly quickly: the execution time of the training phase is on the order of seconds, while classification is on the order of tenths of a second.

Surface Mount Technology Line Optimisation using Modified k-means with Feature Weight Constraints

The common setup problem for chip mounters in a Surface Mount Technology (SMT) line is to group different device models, each of them consisting of different components, into minimum number of clusters where each cluster has a maximum component size. This type of setup problem is classified as a clustering problem with feature weight constraints which cannot be fulfilled by traditional data clustering. In this study, we introduce some vital modifications on standard k-means algorithm such that it can incorporate feature weight constraints adapted from cluster size constraints. We also propose a modification to the elbow method to determine the number of clusters of the clustering problem with feature weight constraints. The results show that the proposed algorithm (modified k-means with feature weight constraints) is able to fulfill the feature weight constraints and solve the common setup problem. The results verify the proposed algorithm has superior performance over standard k-means algorithm in clustering problem with feature weight constraints. For the common setup problem on a given data set, the analysis shows that 1000 runs of the proposed algorithm implemented using MATLAB is able to obtain at least one valid clustering result within a reasonable run time.

Enhanced Firefly-K-Means Clustering with Adaptive Mutation and Central Limit Theorem for Automatic Clustering of High-Dimensional Datasets

Metaheuristic algorithms have been hybridized with the standard K-means to address the latter’s challenges in finding a solution to automatic clustering problems. However, the distance calculations required in the standard K-means phase of the hybrid clustering algorithms increase as the number of clusters increases, and the associated computational cost rises in proportion to the dataset dimensionality. The use of the standard K-means algorithm in the metaheuristic-based K-means hybrid algorithm for the automatic clustering of high-dimensional real-world datasets poses a great challenge to the clustering performance of the resultant hybrid algorithms in terms of computational cost. Reducing the computation time required in the K-means phase of the hybrid algorithm for the automatic clustering of high-dimensional datasets will inevitably reduce the algorithm’s complexity. In this paper, a preprocessing phase is introduced into the K-means phase of an improved firefly-based K-means hybrid algorithm using the concept of the central limit theorem to partition the high-dimensional dataset into subgroups of randomly formed subsets on which the K-means algorithm is applied to obtain representative cluster centers for the final clustering procedure. The enhanced firefly algorithm (FA) is hybridized with the CLT-based K-means algorithm to automatically determine the optimum number of cluster centroids and generate corresponding optimum initial cluster centroids for the K-means algorithm to achieve optimal global convergence. Twenty high-dimensional datasets from the UCI machine learning repository are used to investigate the performance of the proposed algorithm. The empirical results indicate that the hybrid FA-K-means clustering method demonstrates statistically significant superiority in the employed performance measures and reducing computation time cost for clustering high-dimensional dataset problems, compared to other advanced hybrid search variants.

Clustering using matrix product states

The matrix product state has been demonstrated to be able to explore the most relevant portion of the exponentially large quantum Hilbert space and find accurate solutions for one-dimensional interacting quantum many-body systems. Inspired by this success, here we propose a clustering algorithm based on the matrix product state, which first maps the classical data into quantum states represented as matrix product states, and then minimizes the loss function using a variational matrix product states algorithm in the enlarged space. We demonstrate this algorithm by applying it to several commonly used machine learning data sets, showing that this algorithm could reach higher learning precision and that it is less likely to be trapped in local minima compared to the standard K-means algorithm. We also show that this algorithm can achieve state-of-the-art learning precision on popular computer vision data sets when used in combination with better initialization schemes.

K-Means image segmentation using Mumford–Shah model

k-Means (KM) is well known for its ease of implementation as a clustering technique. It has been applied for color quantization in RGB, YUV, hyperspectral image, Lab, and other spaces, but this leads to fragmented segments as the pixels are clustered only in the color space without considering connectivity. The problem has been attacked by adding connectivity constraints, or using joint color and spatial features (r, g, b, x, y), which prevent fragmented and nonconvex segments. However, it does not take into account the complexity of the shape itself. The Mumford–Shah model has been earlier used to overcome this problem but with slow and complex mathematical optimization algorithms. We integrate Mumford–Shah model directly into KM context and construct a fast and simple implementation of the algorithm. The proposed approach uses standard KM algorithm with distance function derived from Mumford–Shah model so that it optimizes both the content and the shape of the segments jointly. We demonstrate by experiments that the proposed algorithm provides better results than comparative methods when compared using various error evaluation criteria. The algorithm is applied on 100 images in the Weizmann dataset and two remote sensing images.

Clustering Analysis using an Unsupervised Machine Learning Method

Artificial Intelligence (AI) and Machine Learning (ML), which are becoming a part of interest rapidly for various researchers. ML is the field of Computer Science study, which gives capability to learn without being absolutely programmed. This work focuses on the standard k-means clustering algorithm and analysis the shortcomings of the standard k-means algorithm. The k-means clustering algorithm calculates the distance between each data object and not all cluster centres in every iteration, which makes the efficiency of clustering is high. In this work, we have to try to improve the k-means algorithm to solve simple data to store some information in every iteration, which is to be used in the next interaction. This method avoids computing distance of data object to the cluster centre repeatedly, saving the running time. An experimental result shows the enhanced speed of clustering, accuracy, reducing the computational complexity of the k-means. In this, we have work on iris dataset extracted from Kaggle.

Comprehensive Review of K-Means Clustering Algorithms

This paper presents a comprehensive review of existing techniques of k-means clustering algorithms made at various times. The k-means algorithm is aimed at partitioning objects or points to be analyzed into well separated clusters. There are different algorithms for k-means clustering of objects such as traditional k-means algorithm, standard k-means algorithm, basic k-means algorithm and the conventional k-means algorithm, this are perhaps the most widely used versions of the k-means algorithms. These algorithms uses the Euclidean distance as its metric and minimum distance rule approach by assigning each data points (objects) to its closest centroids.

Unsupervised color image segmentation: A case of RGB histogram based K-means clustering initialization.

Color-based image segmentation classifies pixels of digital images in numerous groups for further analysis in computer vision, pattern recognition, image understanding, and image processing applications. Various algorithms have been developed for image segmentation, but clustering algorithms play an important role in the segmentation of digital images. This paper presents a novel and adaptive initialization approach to determine the number of clusters and find the initial central points of clusters for the standard K-means algorithm to solve the segmentation problem of color images. The presented scheme uses a scanning procedure of the paired Red, Green, and Blue (RGB) color-channel histograms for determining the most salient modes in every histogram. Next, the histogram thresholding is applied and a search in every histogram mode is performed to accomplish RGB pairs. These RGB pairs are used as the initial cluster centers and cluster numbers that clustered each pixel into the appropriate region for generating the homogeneous regions. The proposed technique determines the best initialization parameters for the conventional K-means clustering technique. In this paper, the proposed approach was compared with various unsupervised image segmentation techniques on various image segmentation benchmarks. Furthermore, we made use of a ranking approach inspired by the Evaluation Based on Distance from Average Solution (EDAS) method to account for segmentation integrity. The experimental results show that the proposed technique outperforms the other existing clustering techniques by optimizing the segmentation quality and possibly reducing the classification error.

Quality and size assessment of quantized images using K-Means++ clustering

In this paper, an amended K-Means algorithm called K-Means++ is implemented for color quantization. K-Means++ is an improvement to the K-Means algorithm in order to surmount the random selection of the initial centroids. The main advantage of K-Means++ is the centroids chosen are distributed over the data such that it reduces the sum of squared errors (SSE). K-Means++ algorithm is used to analyze the color distribution of an image and create the color palette for transforming to a better quantized image compared to the standard K-Means algorithm. The tests were conducted on several popular true color images with different numbers of K value: 32, 64, 128, and 256. The results show that K-Means++ clustering algorithm yields higher PSNR values and lower file size compared to K-Means algorithm; 2.58% and 1.05%. It is envisaged that this clustering algorithm will benefit in many applications such as document clustering, market segmentation, image compression and image segmentation because it produces accurate and stable results.

Parallel implementation of the greedy heuristic clustering algorithms

Authors propose parallel greedy heuristic k-means clustering algorithms for implementation on the graphical processing units (GPU) for solving large-scale problems. The computational experiments illustrate high performance of the GPUs in comparison with running the greedy heuristic algorithms on a central processor unit which is especially significant in the case of big datasets and bug numbers of clusters. The efficiency of the greedy heuristic algorithms in comparison with the standard k-means algorithm remains.

Validity index based improvisation in reproducibility of load profiling outcome

Under competitive market environment, the knowledge of load variation and its behavioural aspect are essential for the survival of electric supply utility. Load profiling is a useful tool to acquire it, and k-means clustering is a crisp, robust and widely proposed method. However, the multiple runs of a k-means algorithm for the same problem do not result in a deterministic outcome because of its dependency on initialisation. This study proposes an improved k-means algorithm to reduce the stochasticity in the outcome. The methodology combines clustering and qualitative validation processes and extracts the best outcome based on several validity indices that take into account cohesion and separation. The external cluster validity indices are innovatively used for comparison of the proposed method with the k-means algorithm. If the same problem is repeatedly executed, the variation in the outcome is lesser as compared to the standard k-means algorithm, thus improving the repeatability and overall quality of the clustering. The 24 real-life data-sets of load patterns of distribution feeders of the Central Indian state discom have been used for the study. The results show that proposed methodology reduces the stochasticity in the outcome significantly.

Optimized K-Means Clustering Model based on Gap Statistic

Big data has become famous to process, store and manage massive volumes of data. Clustering is an essential phase in big data analysis for many real-life application areas uses clustering methodology for result analysis. The data clustered sets have become a challenging issue in the field of big data analytics. Among all clustering algorithm, the K-means algorithm is the most widely used unsupervised clustering approach as seen from past. The K-means algorithm is the best adapted for deciding similarities between objects based on distance measures with small datasets. Existing clustering algorithms require scalable solutions to manage large datasets. However, for a particular domain-specific problem the initial selection of K is still a significant concern. In this paper, an optimized clustering approach presented which is calculated the optimal number of clusters (k) for specific domain problems. The proposed approach is an optimal solution based on the cluster performance measure analysis based on gab statistic. By observation, the experimental results prove that the proposed model can efficiently enhance the speed of the clustering process and accuracy by reducing the computational complexity of the standard k-means algorithm which achieves 76.3%.

A Fast Partitional Clustering Algorithm based on Nearest Neighbours Heuristics

K-means, along with its several other variants, is the most widely used family of partitional clustering algorithms. Generally speaking, this family of algorithm starts by initializing a number of data points as cluster centres, and then iteratively refines these cluster centres based on the current partition of the dataset. Given a set of cluster centres, inducing the partition over the dataset involves finding the nearest (or most similar) cluster centre for each data point, which is an O(NK) operation, N and K being the number of data points and the number of clusters, respectively. In our proposed approach, we avoid the explicit computation of these distances for the case of sparse vectors, e.g. documents, by utilizing a fundamental operation, namely TOP(x), which gives a list of the top most similar vectors with respect to the vector x. A standard way to store sparse vectors and retrieve the top most similar ones given a query vector, is with the help of the inverted list data structure. In our proposed method, we use the TOP(x) function to first select cluster centres that are likely to be dissimilar to each other. Secondly, to obtain the partition during each iteration of K-means, we avoid the explicit computation of the pair-wise similarities between the centroid and the non-centroid vectors. Thirdly, we avoid recomputation of the cluster centroids by adopting a centrality based heuristic. We demonstrate the effectiveness of our proposed algorithm on TREC-2011 Microblog dataset, a large collection of about 14M tweets. Our experiments demonstrate that our proposed method is about 35x faster and produces more effective clusters in comparison to the standard K-means algorithm.

Cuckoo, Bat and Krill Herd based k-means++ clustering algorithms

Traditional k-means clustering algorithm is sensitive to the choice of initial cluster centers and leads to local optimal results. k-means++ is a hybrid k-means clustering algorithm which specifies the procedure to initialize the cluster centers before proceeding with the standard k-means algorithm. Inspired by nature, some contemporary optimization techniques such as Cuckoo, Bat and Krill Herd algorithms etc., are used for optimization as they mimic the swarming behaviour and allows to cooperatively move towards an optimal objective within a reasonable time. In this paper, these nature-inspired techniques are used for optimizing k-means++ clustering algorithm to enhance clustering quality and generate new hybrids of unprecedented performance. The results of the evaluation experiments on the integration of nature-inspired optimization methods with k-means++ algorithm are reported.

Unsupervised learning by cluster quality optimization

Most clustering algorithms are designed to minimize a distortion measure which quantifies how far the elements of the clusters are from their respective centroids. The assessment of the results is often carried out with the help of cluster quality measures which take into account the compactness and separation of the clusters. However, these measures are not amenable to optimization because they are not differentiable with respect to the centroids even for a given set of clusters. Here we propose a differentiable cluster quality measure, and an associated clustering algorithm to optimize it. It turns out that the standard k-means algorithm is a special case of our method. Experimental results are reported with both synthetic and real datasets, which demonstrate the performance of our approach with respect to several standard quantitative measures.

A big data approach for clustering and calibration of link fundamental diagrams for large-scale network simulation applications

Existing methods for calibrating link fundamental diagrams (FDs) often focus on a limited number of links and use grouping strategies that are largely dependent on roadway physical attributes alone. In this study, we propose a big data-driven two-stage clustering framework to calibrate link FDs for freeway networks. The first stage captures, under normal traffic state, the variations of link FDs over multiple days based on which links are clustered in the second stage. Two methods, i.e. the standard k-means algorithm combined with hierarchical clustering and a modified hierarchical clustering based on the Fréchet distance, are applied in the first stage to obtain the FD parameter matrix for each link. The calibrated matrices are input into the second stage where the modified hierarchical clustering is re-employed asa static approach resulting in multiple clusters of links. To further consider the variations of link FDs, the static approach is extended by modifying the similarity measure through the principle component analysis (PCA). The resulting multivariate time-series clustering models the distributions of the FD parameters asa dynamic approach. The proposed framework is applied on the Melbourne freeway network using one-year worth of loop detector data. Results have shown that (a) similar roadway physical attributes do not necessarily result in similar link FDs, (b) the connectivity-based approach performs better in clustering link FDs as compared with the centroid-based approach, and (c) the proposed framework helps achieving a better understanding of the spatial distribution of links with similar FDs and the associated variations and distributions of the FD parameters.

A Big Data Approach for Clustering and Calibration of Link Fundamental Diagrams for Large-Scale Network Simulation Applications

Existing methods for calibrating link fundamental diagrams (FDs) often focus on a limited number of links and use grouping strategies that are largely dependent on roadway physical attributes alone. In this study, we propose a big data-driven two-stage clustering framework to calibrate link FDs for freeway networks. The first stage captures, under normal traffic state, the variations of link FDs over multiple days based on which links are clustered in the second stage. Two methods, i.e. the standard k-means algorithm combined with hierarchical clustering and a modified hierarchical clustering based on the Fréchet distance, are applied in the first stage to obtain the FD parameter matrix for each link. The calibrated matrices are input into the second stage where the modified hierarchical clustering is re-employed as a static approach resulting in multiple clusters of links. To further consider the variations of link FDs, the static approach is extended by modifying the similarity measure through the principle component analysis (PCA). The resulting multivariate time-series clustering models the distributions of the FD parameters as a dynamic approach. The proposed framework is applied on the Melbourne freeway network using one-year worth of loop detector data. Results have shown that (a) similar roadway physical attributes do not necessarily result in similar link FDs, (b) the connectivity-based approach performs better in clustering link FDs as compared with the centroid-based approach, and (c) the proposed framework helps achieving a better understanding of the spatial distribution of links with similar FDs and the associated variations and distributions of the FD parameters.

Package balancing k-means algorithm for physical distribution

As the express delivery service is flourishing, it is a challenge to assign a set of tasks to a set of carriers, as it has to aggregate the neighbour tasks to a carrier and keep the loads of carriers balanced simultaneously. K-means clustering algorithm is an effective way to split the delivery tasks. However, it cannot solve the load balancing problem. This paper proposes an extended clustering algorithm - package balancing k-means. It adding the weight metric to the standard k-means algorithm to achieve load balancing. Analyses and experiments show that package balancing k-means can solve the load balancing clustering problem efficiently. Moreover, experiments show that the extended algorithm can achieve faster convergence.

Package balancing k-means algorithm for physical distribution

As the express delivery service is flourishing, it is a challenge to assign a set of tasks to a set of carriers, as it has to aggregate the neighbour tasks to a carrier and keep the loads of carriers balanced simultaneously. K-means clustering algorithm is an effective way to split the delivery tasks. However, it cannot solve the load balancing problem. This paper proposes an extended clustering algorithm - package balancing k-means. It adding the weight metric to the standard k-means algorithm to achieve load balancing. Analyses and experiments show that package balancing k-means can solve the load balancing clustering problem efficiently. Moreover, experiments show that the extended algorithm can achieve faster convergence.