Incremental Clustering Algorithm Research Articles

Planning, Design, and Execution of a Quantum Incremental Clustering Algorithm System for the Examination of Unsupervised Data

The exponential growth of data in various domains has underscored the need for efficient unsupervised learning algorithms to discover hidden patterns and structures within large datasets. Traditional clustering algorithms often face challenges in handling dynamic datasets and require periodic retraining, making them less suitable for real-time applications. In response to these challenges, this research introduces a Quantum Incremental Clustering Algorithm System (QICAS) designed to adapt to evolving datasets and provide timely insights into the underlying structures. The proposed QICAS is based on quantum computing principles, leveraging the unique properties of qubits and quantum superposition to perform clustering tasks more efficiently than classical counterparts. The research focuses on the planning, design, and execution of the QICAS, ensuring a comprehensive understanding of the algorithm's capabilities and performance.

An incremental clustering algorithm based on semantic concepts

An incremental clustering algorithm based on semantic concepts

Key grids based batch-incremental CLIQUE clustering algorithm considering cluster structure changes

In the network environment, data from various industries is dynamic and large-scale. Traditional clustering algorithms struggle to effectively utilize existing clustering results when faced with continuously evolving data, which makes the incremental grid-based clustering highly regarded. However, the existing incremental grid-based clustering algorithms fail to adequately consider the impact of newly added data on the original cluster structure. To address this issue, the key grids based batch-incremental CLIQUE clustering algorithm is proposed. The algorithm designates the incremental data mapping grids, which are or their neighbour girds are mixed with original data, as key grids to fully consider the cluster structure changes caused by the incremental data. Moreover, the cluster similarity coefficient based on grid features is introduced to measure density differences between the incremental data and the original clusters, and the cluster membership degree is defined to further consider the cluster membership of boundary sparse grid data and the identification of noise points. All of which ensures that the algorithm can adaptively create, merge or split clusters with the arrival of new data. Experimental results show that the proposed algorithm can adaptively adjust the cluster structure during incremental clustering, outperforming in accuracy and efficiency when clustering large-scale, dynamically changing data.

An Incremental Clustering Algorithm for Arbitrary Shaped in IoT Data

An Incremental Clustering Algorithm for Arbitrary Shaped in IoT Data

Incremental density clustering framework based on dynamic microlocal clusters

With the prevailing development of the internet and sensors, various streaming raw data are generated continually. However, traditional clustering algorithms are unfavorable for discovering the underlying patterns of incremental data in time; clustering accuracy cannot be assured if fixed parameters clustering algorithms are used to handle incremental data. In this paper, an Incremental-Density-Micro-Clustering (IDMC) framework is proposed to address this concern. To reduce the succeeding clustering computation, we design the Dynamic-microlocal-clustering method to merge samples from streaming data into dynamic microlocal clusters. Beyond that, the Density-center-based neighborhood search method is proposed for periodically merging microlocal clusters to global clusters automatically; at the same time, these global clusters are updated by the Dynamic-cluster-increasing method with data streaming in each period. In this way, IDMC processes sensor data with less computational time and memory, improves the clustering performance, and simplifies the parameter choosing in conventional and stream data clustering. Finally, experiments are conducted to validate the proposed clustering framework on UCI datasets and streaming data generated by IoT sensors. As a result, this work advances the state-of-the-art of incremental clustering algorithms in the field of sensors’ streaming data analysis.

Statement of Retraction: Incremental Outlier Feature Clustering Algorithm in Blockchain Networks Based on Big Data Analysis

Statement of Retraction: Incremental Outlier Feature Clustering Algorithm in Blockchain Networks Based on Big Data Analysis

A big data exploration approach to exploit in-vehicle data for smart road maintenance

In modern Smart Cities, pervasive collection of sensor-based and IoT data streams is a challenging opportunity for improving mobility resilience. Among the potential applications, sensor-based data streams provide valuable information about the quality of the area-wide road surface. Modern vehicle black boxes are also able to estimate the type of anomaly (e.g., bump, hole, rough ground, depression), based on real-time analysis of acceleration data streams. Road maintainers may use all this information to improve monitoring and maintenance activities. However, the volume of data streams, the variety of road network and different degrees of seriousness of detected anomalies call for methods to support maintainers in the exploration of available data. To this aim, in this paper, we propose a methodological approach, based on big data exploration techniques. The approach is grounded on: (i) a multi-dimensional model, apt to organise data streams according to different dimensions and enable data exploration; (ii) data summarisation techniques, based on an incremental clustering algorithm, to simplify the overall view over massive data streams and to cope with their dynamic nature; (iii) a measure of relevance, to focus the attention on road portions that present critical conditions. The innovative contributions regard the formalisation of the exploration methodology, the definition of exploration scenarios, based on road maintainers’ goals and the measure of relevance, and an extensive experimentation on a real world case study, addressed in a research project on smart and resilient mobility. Experimental results show how relevance evaluation is able to efficiently attract the road maintainers’ attention on road portions that present the most critical conditions and the proposed incremental clustering algorithm outperforms existing algorithms in the literature.

Incremental Deinterleaving of Radar Emitters

In this paper, we present a new paradigm for the deinterleaving of radar signals in electronic support systems. Traditional systems observe the electromagnetic spectrum for a longer period of time and perform the deinterleaving operation batch-wise. This paper presents a new approach, where the deinterleaving is performed after each detected radar pulse, which we call pulse-to-pulse incremental deinterleaving. It enables minimum reaction time of the system for the detection of radar emitters. For that purpose the paper presents three incremental clustering algorithms based on DBSCAN, Adaptive Resonance Theory and the Leader approach. The algorithms are evaluated using simulated PDW data of realistic complex radar scenes. The results show, that incremental approaches can achieve a similar accuracy as non-incremental methods, but have much faster reaction times to the occurrence of new emitters.

Dynamic conditional score model-based weighted incremental fuzzy clustering of consumer power load data

This study proposes a weighted incremental fuzzy C-mean power load clustering algorithm based on the dynamic conditional score model to solve the problems that the predominant power load data mining method only captures the mean characteristics of time series and ignores the heteroscedasticity and sequence correlations. Consequently, this method has unsatisfactory time series clustering and low clustering accuracy. A dynamic conditional score model is constructed to analyze and extract statistical characteristic parameters of a time series to calculate the autocorrelation value of the parameter series. A weighted fuzzy C-mean clustering analysis is performed, and the obtained data weight information is used as input for incremental clustering to improve the clustering accuracy. The DCS model parameter dataset and data weight information are combined, and the clustering analysis of the consumer power load data stream is performed. The power load time series of different companies is given, and the clustering validity indices are defined for the performance analysis to verify the proposed clustering algorithm. The experimental results show that the proposed algorithm achieves satisfactory clustering and improves the performance.

Incremental clustering algorithm based on representative points and covariance for large data

Incremental clustering algorithm based on representative points and covariance for large data

Incremental clustering algorithm based on representative points and covariance for large data

Incremental clustering algorithm based on representative points and covariance for large data

A novel optimization approach towards improving separability of clusters

The objective functions in optimization models of the sum-of-squares clustering problem reflect intra-cluster similarity and inter-cluster dissimilarities and in general, optimal values of these functions can be considered as appropriate measures for compactness of clusters. However, the use of the objective function alone may not lead to the finding of separable clusters. To address this shortcoming in existing models for clustering, we develop a new optimization model where the objective function is represented as a sum of two terms reflecting the compactness and separability of clusters. Based on this model we develop a two-phase incremental clustering algorithm. In the first phase, the clustering function is minimized to find compact clusters and in the second phase, a new model is applied to improve the separability of clusters. The Davies–Bouldin cluster validity index is applied as an additional measure to compare the compactness of clusters and silhouette coefficients are used to estimate the separability of clusters. The performance of the proposed algorithm is demonstrated and compared with that of four other algorithms using synthetic and real-world data sets. Numerical results clearly show that in comparison with other algorithms the new algorithm is able to find clusters with better separability and similar compactness.

Research on incremental clustering algorithm for big data

Abstract As the scale of data becomes larger and larger, clustering processing, a key step in data mining, has important practical significance. Aiming at the problems of time consumption and high clustering errors when the current clustering algorithms deal with massive and dynamic big data, an incremental clustering algorithm is proposed by taking big data as the research object. By exploring the attribute characteristics of big data, four characteristics such as scale, diversity, high speed and value are summarised. For large-scale data streams that have multiple attributes and are acquired one by one, optimise the setting method of the K-means clustering algorithm category centre point, combine the K-means clustering algorithm and the Kalman filter algorithm and measure the distance between data point pairs. Instead of Mahalanobis distance, an incremental clustering algorithm suitable for big data is constructed. Five data sets are selected to carry out example analysis. The results of the algorithm are verified by the algorithm. The proposed algorithm has obvious advantages in the incremental clustering effect of big data. At the same time, it also has efficient and stable computing performance, which meets the expected design requirements and goals.

Context-aware incremental clustering of alerts in monitoring systems

The highly complex nature of today’s modern hybrid IT applications continues to present an increasing challenge for operation teams relying on traditional monitoring approaches. In monitoring systems, incidents occur frequently due to a variety of causes, from updates to software and hardware, to changes in operation environment. These incidents could significantly degrade the system’s availability and customers’ satisfaction. In many cases, investigating an incident in such an environment could feel like looking for a needle in a haystack - and you may not even know how the needle looks like until you see it. In that regard, one of the main challenges is how to efficiently analyze multiple sets of alert messages stemming from disparate monitoring tools and collectors across the application stack, in real-time. Such an analysis can provide trustworthy detection of system states at various critical points, thus helping teams to detect, frame, analyze and resolve incidents or failures in a relatively short time, especially if an accurate system’s topological dependencies are absent. In this work, we suggest a new approach to determining relations among alerts – forming “events”. The suggested approach directly models the event’s likelihood, by first embedding alerts’ corresponding metrics into a common latent space where the distance among metrics can be naturally defined, using a word2vec model, and then cluster alerts by employing a tailored incremental clustering algorithm. The suggested approach allows controlling the trade-off between the model’s sensitivity to clusters’ noise-robustness, thus spanning a wide range of clustering mechanisms, as well as adapting clusters’ outcomes to the level and properties of the noise expected in input data.

ITL-IDS: Incremental Transfer Learning for Intrusion Detection Systems

Utilizing machine learning methods to detect intrusion into computer networks is a trending topic in information security research. The limitation of labeled samples is one of the challenges in this area. This challenge makes it difficult to build accurate learning models for intrusion detection. Transfer learning is one of the methods to counter such a challenge in machine learning topics. On the other hand, the emergence of new technologies and applications might bring new vulnerabilities to computer networks. Therefore, the learning process cannot occur all at once. Incremental learning is a practical standpoint to confront this challenge. This research presents a new framework for intrusion detection systems called ITL-IDS that can potentially start learning in a network without prior knowledge. It begins with an incremental clustering algorithm to detect clusters’ numbers and shape without prior assumptions about the attacks. The outcomes are candidates to transfer knowledge between other instances of ITL-IDS. In each iteration, transfer learning provides target environments with incremental knowledge. Our evaluation shows that this method can combine incremental and transfer learning to identify new attacks.

Analysis of the Running Ability Mining Model of Football Trainers Based on Dynamic Incremental Clustering Algorithm.

Fast-running ability is a very important basic quality of football players. However, players are dynamic. It is difficult for coaches to grasp the running speed, instantaneous acceleration, and other indicators of small athletes in real time with the naked eye. Therefore, to accurately test the performance of athletes in fast-running ability, this paper studies the running ability mining model of football coaches based on the dynamic incremental clustering algorithm. According to scientific procedures and methods, the evaluation model and standard of running ability of Chinese elite female football players are established. The effectiveness of the model is 0.83, as verified by the standard recognition method, which shows that the evaluation model is efficient. The research considers the denoising of the original data. The model has rich data and standard test methods and procedures. It can be used as a measure of the running ability of China's elite female football players in a certain period and range. The research solves the problem of the insufficient running ability of domestic football players. It provides an important reference for training the next generation of excellent national football players.

NGIA: A novel Greedy Incremental Alignment based algorithm for gene sequence clustering

Gene sequence clustering is very basic and important in computational biology and bioinformatics for the study of phylogenetic relationships and gene function prediction, etc. With the rapid growth of the amount of biological data (gene/protein sequences), gene sequence clustering algorithms face more challenges in low precision and efficiency. The growing redundant sequences in gene sequence databases usually contribute to the increasing memory and computing demand for most clustering methods. For example, the original greedy incremental alignment-based (GIA) clustering algorithm obtains high precision clustering results, but with very poor efficiency. Efficient greedy incremental clustering algorithms have been developed with a cost of precision reduction, which usually trade clustering precision off for speed improvement. Algorithms with a better balance between precision and speed are needed. This paper proposes a novel Greedy Incremental Alignment-based algorithm called nGIA for gene clustering with high efficiency and precision. nGIA consists of a pre-filter, a modified short word filter, a new data packing strategy, a modified greedy incremental method, and is parallelized via GPU. The experimental evaluations on four independent datasets show that the proposed tool can cluster datasets with high precisions of 99.99%. Compared with the results of CD-HIT, Vsearch, and Uclust, nGIA is on average 13.6x, 6.2x, and 1.7x faster. In addition, we have developed a multi-node version to handle large data sets. The strong scalability test shows that the multi-node version of nGIA can scale up to 32 threads with a 31% parallel efficiency. The software is available at https://github.com/SIAT-HPCC/gene-sequence-clustering.

Discovering Overlapping Communities in Dynamic Networks Based on Cascade Information Diffusion

Complex networks in real world are always in the state of evolution and composed of numerous overlapping communities. The discovery of overlapping communities in dynamic networks plays an important role in community detection research. In recent years, methods based on incremental clustering have become increasingly popular owing to their high efficiency. However, few of them can deal with communities that are both overlapping and dynamic. In this article, we propose an incremental clustering algorithm for discovering overlapping communities in dynamic networks. In the initial snapshot of a dynamic network, a degree-based seed selection strategy with concise and effective rules is employed to obtain stable and high-quality overlapping communities, in which the degree of nodes is the number of their neighboring nodes in the subgraph composed of free nodes. In the subsequent snapshots, a four-staged framework based on cascade information diffusion is proposed to update the communities incrementally. In this framework, a cascade information diffusion model is used to simulate the evolution of communities and then the fitness of nodes to the communities they belong to is updated based on node similarity. Experiments conducted on both real-world and artificial datasets show that the proposed algorithm can discover overlapping communities in dynamic networks effectively and outperform to the state-of-art baseline algorithms.

RETRACTED ARTICLE: Incremental Outlier Feature Clustering Algorithm in Blockchain Networks Based on Big Data Analysis

Outlier detection in data mining aims at identifying anomalous observations known as outliers. The standard communication network incremental outlier feature clustering technique has a low clustering impact, requires several iterations, and takes a long clustering time. A novel incremental outlier based on big data analysis is proposed to overcome these problems for communicating networks in a Blockchain environment. Through big data analysis, the Point feature clustering algorithm uses the kernel density estimation technique with Gaussian kernel function to estimate the incremental outlier density of the communication network in the Blockchain framework. It mines the incremental outlier and applies the empirical mode based on the outlier mined for optimal outcome. The decomposition technique is used to extract outlier features, give weights to the features, and calculate the outlier degree and clusters. According to the results, the communication network, based on the Blockchain environment, may be clustered for incremental outlier features, and then outliers are ideally detected optimally. The simulation of the proposed technique is performed on three types of datasets. The simulation to form a cluster with the proposed method takes around 10 s of clustering time. The simulation results reveal that the proposed incremental outlier feature clustering (OFC) technique has a better outlier feature clustering impact on Blockchain-based networks, and requires a fewer iterations due to low computational and space complexity than other existing clustering techniques.

Analysis of Chinese Patents associated with Incremental Clustering Algorithms: A Review

With the advent of Internet-of-Things (IoT) and overall Information-Technology world, an enormous amount of data is getting generated dynamically and in real-time mode, in almost all domains of research and application systems. Such huge data has embedded patterns and hidden information to extract and learn. This learning is incremental in nature for all involved entities and users, as the data is growing exponentially in real-time. To achieve learning from such dynamic data sources, incremental clustering algorithms are used mandatorily. This mandate has given rise to increased patents related to incremental clustering concept, which is primarily a significant part of Machine Learning field. In this paper, we contribute to the in-progress discussion on the use of intellectual property resources, particularly patents related to machine learning, incremental clustering, incremental learning with a special focus to country China. Due consideration of the prior art search, the author found that China the country of registration of the application extensively contributes to the intellectual property related to incremental clustering domain hence felt the need to undertake this detailed patent analysis about this topic. We hope all readers, research scholars will be benefited with the latest research presented in this paper pertaining to various patents in the advanced areas of computer engineering.