Data Mining Clustering Algorithm Research Articles

Analysis of the opportunities and challenges of information technology for enterprise development strategy based on big data technology

Abstract Based on big data information technology, this paper analyzes the opportunities and challenges of business development strategies. The clustering algorithm and tree model algorithm in data mining is analyzed. In order to effectively solve the problem of big data classification in consumer-oriented enterprises, the Kmeans clustering algorithm and XGBoost algorithm in the two previous models are integrated to effectively avoid the problem of over-fitting when the models are used alone. The opportunities and challenges in the current stage of business development strategy are analyzed separately. The Kmeans-XGBoost algorithm is used to analyze the pricing and output of the enterprise for prediction. It is shown that the prediction curves of the Kmeans-XGBoost model basically match the actual values, and the confidence interval range is expanding from [3694.879,7202.897] to [2211.819,8406.462]. Meanwhile, the errors of enterprise output prediction under different algorithms are analyzed. The error rate weighted by Kmeans-XGBoost mean is 42.63, which is lower than the traditional model prediction error in 4.

An enhanced self-learning-based clustering scheme for real-time traffic data distribution in wireless networks

The process of examining the data flow over the internet to identify abnormalities in wireless network performance is known as network traffic analysis. When analyzing network traffic data, traffic classification becomes an important task. The traffic data classification is used to determine whether data in network traffic is in real-time or not. This analysis controls network traffic data in a network and allows for efficient network performance improvement. Real-time and non-real-time data are effectively classified from the given input data set using data mining clustering and classification algorithms. The proposed work focuses on the performance of traffic data classification with high clustering accuracy and low Classification Time (CT). This research work is carried out to fill the gap in the existing network traffic classification algorithms. However, the traffic data classification remained unaddressed for performing the network traffic analysis effectively. Then, we proposed an Enhanced Self-Learning-based Clustering Scheme (ESLCS) using an enhanced unsupervised algorithm and adaptive seeding approach to improve the classification accuracy while performing the real-time traffic data distribution in wireless networks. Test-bed results demonstrate that the proposed model enhances the clustering accuracy and True Positive Rate (TPR) effectively as well as reduces the CT time and Communication Overhead (CO) substantially to compare with the peer-existing routing techniques.

Enhancement of Data Mining Clustering Algorithm Based On Innovative Artificial Intelligence Approach

Enhancement of Data Mining Clustering Algorithm Based On Innovative Artificial Intelligence Approach

Fast spectral clustering with self-adapted bipartite graph learning

Spectral Clustering (SC) is a widespread used clustering algorithm in data mining, image processing, etc. It is a graph-based algorithm capable of handling arbitrarily distributed data. However, the distances of all samples in the high-dimensional space tend to be equal, so the similarity matrix of SC may not be reasonable in high-dimensional data. In addition, the similarity matrix and clustering results in SC are performed in two steps. To solve these problems, a novel joint clustering method called fast spectral clustering with self-adapted bipartite graph learning (FSBGL), is proposed. It is capable of obtaining low-dimensional representations from high dimensional data, thus decreasing the complexity and increase the efficiency of the algorithm. In contrast to traditional spectral clustering algorithms that obtain clustering results in a two-step process, FSBGL obtains clustering results directly from the concatenated components of the optimized similarity matrix while learning the optimized bipartite graph. This eliminates the effect of performing these two steps separately in SC. In effect, a more discriminative low-dimensional representation may be derived from the adaptively learned bipartite graph, while the better low-dimensional representation can continue to be used to learn the structure of the graph. The learning of the bipartite graph alternates with the iteration of the low-dimensional representation, which allows the algorithm to obtain more accurate clustering results. Furthermore, by means of a low-dimensional representation on the basis of fast spectral embedding, the algorithm has better performance on some large-scale datasets. The results of the experiment indicate that the FSBGL is better than other comparative methods in various data sets.

Abnormal Operation Detection of Access Equipment in Dispatching Exchange Network Based on Data Mining

To meet the requirements of a power dispatching service, a dispatching switching network has been widely used instead of circuit switching technology. However, with the growth of the number of access devices, the problem of network security is becoming more and more serious. At present, the commonly used access equipment abnormal operation detection method mainly relies on a two-stage clustering method to identify the abnormal operation state, but the processing effect is not good in the face of a large number of data, resulting in a low F1 value of the final detection result. Therefore, aiming at the access equipment of the dispatching exchange network, an abnormal operation detection method based on data mining is proposed. Scapy, a network tool, is used to capture the traffic data of access devices, and the collected data are normalized and fused. Data feature information is extracted through time sliding window and incremental calculation mode. With the clustering algorithm and related rules in data mining as the core, the abnormal operation detection method of access equipment is designed to obtain accurate abnormal detection results. Finally, based on the recursive graph structure, the detection results are visualized. The experimental results show that the average F1 value of the detection results of the proposed method is 0. 98, which indicates that the detection results of the proposed method have higher accuracy, and based on this, the network security isolation is carried out, which effectively improves the security of dispatching switching network.

Novel hybrid geometric data perturbation technique by means of sampling data intervals

Privacy preserving data mining has much importance in the data analysis because it maintains confidential or highly sensitive data in the purpose of data analysis. Many perturbation techniques are developed, but do not perform well. There are used for different purposes in various techniques. Though, the reconstruction of original data can be done from the samples of perturbed data. It does not always preserve the distance that exists in the original data samples. During reconstruction of the original data, some of the information is loosed which is again a complex and time consuming task for traditional data mining algorithm. Therefore, a novel hybrid data perturbation technique is introduced that overcomes the above addressed issues. The proposed paper includes a novel data perturbation technique to hybridize the traditional Geometric Data Perturbation Technique along with the Euclidean distance method to preserve privacy. It obtains the distorted or perturbed data which is sampled at different intervals depending on the levels of privacy. The proposed approach attempts to reduce the tradeoffs that persists between privacy levels, accuracy and information loss. The experimental results for this approach is implemented using real world bench mark datasets downloaded from UCI Repositories using three different data mining clustering algorithms. It is shown that clustering accuracy is similar and fairly high for the perturbed data obtained from the proposed approach when matched with the original data set clustering algorithms. The analysis were shown for different privacy levels on different data samples.

A Novel Classification Performance Approach for Remotely Sensed Multispectral Image Data by Using Data Mining Techniques

In order to combine multimedia imagery and multispectral remote sensing data to analyze information, preprocessing becomes a necessary part of it. As one of the most important branches in the field of data analysis, it is widely used in many fields such as classification, regression, missing value filling, and machine learning. As a lazy algorithm, this method requires no prior statistical knowledge and no additional data to train description rules and is easy to implement. This study compares classification algorithm performances of data mining clustering algorithms for remotely sensed multispectral image data using WEKA data mining software. Clustering algorithm selection is very important for data mining classification method-based clustering. The class attribute for remotely sensed multispectral image data is obtained from six different clustering algorithms for classification. Classification algorithm performances computed depending on the data labeling of six different clustering algorithms in terms of correctly classified instances and kappa statistics for seven different classification algorithms. A strategy is developed for selecting the best unsupervised clustering algorithm, among different clustering algorithms, giving the highest supervised classification accuracy in terms of correctly classified instances and kappa statistics for semi supervised classification of remotely-sensed multispectral image data. The performances of seven semi-supervised classification methods assessed depending on six different unsupervised clustering algorithms for supervised classification of remotely sensed multispectral image data. This study determines data free clustering algorithms for classification.

Multidimensional Information Network Big Data Mining Algorithm Relying on Finite Element Analysis.

In recent years, with the rapid development of the Internet, online social networks have been continuously integrated with traditional interpersonal networks and research on information dissemination in social networks has gradually increased. This article studies and analyzes the multidimensional information network big data mining algorithm based on the finite element analysis method. This paper firstly introduces the finite element analysis and calculation process, a finite element data mining simulation application software management system will integrate current data, calculation, and background data into one, then analyzes the data mining clustering algorithm, and conducts an experimental exploration of the influential node mining algorithm in complex networks. The experimental results show that the LIC algorithm is better than the CC algorithm, the DC algorithm, and the BC algorithm; its overall performance is improved by 30%, and the effect is better. The LIC algorithm can effectively and quickly determine the influential nodes, which is helpful for social network analysis.

Clustering Optimization Algorithm for Data Mining Based on Artificial Intelligence Neural Network

Social production and life have become increasingly prominent. Cluster analysis is the basis for further processing of the data. The concept of data mining and the application of neural networks in data mining are introduced. According to the related technology of data mining, this article introduces in detail the two-layer perceptron, backpropagation (BP) neural network, RBF radial basis function network for processing classification problems, and self-organizing map (SOM) self-organizing neural network for unsupervised clustering problems. According to the characteristics of self-adaptive and self-organizing capabilities of these algorithms, we learn and design and implement data mining clustering optimization algorithms. In this paper, the neural network-based data mining process consists of three stages: data preparation, rule extraction, and rule evaluation. This paper studies the teaching-type and decomposition-type rule extraction algorithms. After analyzing the BP decomposition-type algorithm, the correlation method is used to calculate the correlation of the input and output neurons. After sorting by the degree of correlation, the RBF neural network is used for node selection. This can greatly reduce the number of input nodes of the neural network, simplify the network structure, reduce the number of recursive splits of the subnet, and improve calculation efficiency. Taking the model as an example, the training error is calculated through data mining technology and clustering algorithm. Data mining clustering optimization algorithm mainly improves the popular neural network from two aspects: finer model design and model pruning, and simulates model complexity, computational complexity, and errors through simulation experiments. The rate is measured, and finally, the simulation experiment is performed. The results show that the proposed algorithm for differential distributed data mining has higher accuracy and stronger convergence ability and overcomes the shortcomings and shortcomings of several original genetic algorithm optimization neural network data mining models; it can effectively improve the searchability and search accuracy of the algorithm and improve the efficiency of data mining. Accuracy and accuracy have a wide range of applications.

Mixed statistical and data mining analysis of river flow and catchment properties at regional scale

This paper presents an application of the L-moments and L-moment ratio diagrams (LMRD) to the analysis of hydrological data at regional (country) scale. Existing research focuses on two main areas of the analysis: statistical analysis using LMRD and regression analysis. Further research mixes both approaches applying regression analysis to L-moments. Another direction of the research is clustering of the climatic and physiographic catchment properties and its validation using LMRD. However, LMRD plots can be separately used as the clustering domain. It is proposed to decompose the features into some classes, and than present these results on the LMRD. Such plots constitute the source for the clustering. Obtained clusters are then validated against k-means clustering performed in the LMRD diagram domain. Results show that statistical L-moments analysis can be improved with data mining clustering algorithms. Such combination delivers a new perspective for the interpretation of the results. It is shown that clustering in the LMRD domain is consistent with the K-means clustering. It is anther argument showing that L-moments diagrams can be considered as a very powerful and informative tool for hydrologists enabling the comparison on the regional basis with respect to various catchment properties. The method is validated on data consisting of daily river flow data from 290 gauges covering entire Poland.

Rough set based on least dissimilarity normalized index for handling uncertainty during E-learners learning pattern recognition

The determination of e-learners' learning style in an online environment has raised the potential scope of interest as its exact estimation prompts a sensational improvement in the contents of the learning framework and student performance. It requires a deep investigation of the learning habits of the learner. Grouping e-learners together provides a more quantifiable way to analyze the learner's feedback and log files to discriminate them based on their learning style. This is accomplished with the help of clustering algorithms in data mining that aids in determining their learning styles well. The target clusters are analyzed by generating functional patterns or rules using the rule induction algorithms. Most of the existing works in the literature attributed to the elucidation of learning styles fail to address the uncertainty and inconsistency in the learner's characteristics. The RST is an optimal method for analyzing the learner's behavior in this context. Thus, a Rough set based least dissimilarity normalized index (RS-LDNI) is proposed for resolving uncertainty while estimating e-learners' learning patterns. This RS-LNDI used the merits of Maximum Dependency Attributes (MDA) for categorical clustering such that the maximal dependency between attributes can be determined by splitting attributes instead of Roughness. It also adopted categorical data clustering to attain the correlation between attributes that cannot be used for learning style prediction. The experimental results of the RS-LNDI algorithm outperform the demerits of these existing clustering algorithms by utilizing the reduct and equivalence class property of rough set theory.

An Adjusting Strategy after DBSCAN

DBSCAN is a popular density-based clustering algorithm in data mining. However, its principle of first come, first served to deal with the cross-border points of multiple clusters will make some cross-border points not belong to the best clustering. To solve this problem, an adjusting strategy after DBSCAN to change the positions of the points of every cluster’s convex hull is proposed in this paper. At the end, the effectiveness of proposed adjustment strategy is verified by an experiment.

Network Public Opinion Monitoring Based on MapReduce

Public opinion information on the network is abundantly decentralized in Internet, these false ones, easily misleading, not legally binding, it is difficult to deal with. Quickly master information is a prerequisite to deal with the network of public opinion, clustering algorithm in data mining plays an important role in in the collection of information statistics. Disadvantages of classical clustering methods are high resources consumption, low efficiency, high time and space complexity, and difficult to deal with large-scale data processing in massive network. To the network public opinion text as the research object, in-depth study of summarizing the network public opinion monitoring technology based on distributed MapReduce, including parallel technology, improved clustering algorithm, relational database and distributed database construction, which is in order to improve the efficiency of information processing, to reduce the pin.

DELVING INTO BLOOD TRANSFUSIONS DATA THROUGH DATA MINING: A STUDY OF MAIZBHANDARI SHAH EMDADIA BLOOD DONORS GROUP TO SELECT VOLUNTEER BLOOD DONORS EFFICIENTLY

The demand for blood transfusion is rising gradually. Therefore, volunteer blood donors are needed to save the lives of patients. There are lot of volunteer donors in every society. But, the main problem people face is finding such donors at the right time. To locate potential blood donors, we collected data from the Maizbhandari Shah Emdadia blood donors’ group, which consists of 700 active volunteer blood donors. This study aims to choose potential volunteer donors efficiently at the emergency time from the Maizbhandari Shah Emdadia blood donors’ group based on their past data. We have developed two models, namely descriptive and predictive models, using data mining techniques. The descriptive model analyzes data patterns and explores the donors’ behaviour. A data mining clustering algorithm was used to develop the descriptive model. The underlying factors of donors' intention to donate blood were identified and evaluated using this descriptive model. These factors were then utilized to develop the predictive model, which in turn assists to predict whether a donor will donate blood or not during an emergency. The findings of these two models will assist the clinical experts in locating potential volunteer blood donors within the shortest period and thus save valuable lives.

Applying Data Mining to Analysis of Mismatch Between Education and Labor Market in Indonesia

One of the labor problems in Indonesia is the occurrence of mismatch (incompatibility between education and job). The incompatibility of education and job (labor market) can be categorized into several criteria such as vertical mismatch, horizontal mismatch, overqualification, under qualification, over skilling, under skilling, credentialism and economic skills. In several countries, analysis of mismatch between education and labor market can be used with several methods such as the Maximum Likelihood equation, the Mincer-type income equation, and the use of clustering algorithms in data mining. Indonesia can adopt a method of measuring the mismatch between education and labor market which suitable with the condition in Indonesia. The use of the K-Means algorithm or the hierarchical clustering algorithm will be able to see how much mismatch is occurring in Indonesia. Clustering is carried out referring to the Republic of Indonesia’s Presidential Regulation No. 8/2012 and the Indonesian Classification of Work (KBJI).

Enhanced Data Lake Clustering Design based on K-means Algorithm

In recent years, Big Data requirements have evolved. Organizations are trying more than ever to accent their efforts on industrial development of all data at their disposal and move further away from underpinning technologies. After investing around Data Lake concept, organizations must now overhaul their data architecture to face IoT (Internet of Things) and AI (Artificial Intelligence) expansion. Efficient and effective data mapping treatments could serve in understanding the importance of data being transformed and used for decision-making process endorsement. As current relational databases are not able to manage large amounts of data, organizations headed towards NoSQL (Not only Structured Query Language) databases. One such known NoSQL database is MongoDB, which has a high scalability. This article mainly put forward a new data model able to extract, classify, and then map data for the purpose of generating new more structured data that meet organizational needs. This can be carried out by calculating various metadata attributes weights, which are considered as important information. It also processed on data clustering stored into MongoDB. This categorization based on data mining clustering algorithm named K-Means.

Research of Profitable Period for the Jasmine Farmers Based on its Yield Estimation and Market Price using K-Means Clustering Algorithm

Jasmine is the oldest fragrance flower and commercial crop of south India. It is mostly used for garlands, perfumes, cosmetic and medicinal purpose. Its use is only for 24 hours. Jasmine grows successful in all season but farmer does not have the proper guidance to cultivate the jasmine in correct climate to cultivate, month to plant and harvest and best fertile soil to plant the jasmine will help the farmers for best cultivation and productivity of jasmine. But without the knowledge the farmer’s not get the profit of selling the jasmine in all season. In peak season the jasmine flower is high and the price is low in market and in lean season the flower is low but the market price is high. This is the main reason for the research to choose a particular month in lean season to increase the jasmine flower yield to get more profit to the farmers, and the yield estimation, data sets are collected and calculated using K-Means Clustering Algorithm in data mining

Optimizing MSE for Clustering with Balanced Size Constraints

Clustering is to group data so that the observations in the same group are more similar to each other than to those in other groups. k-means is a popular clustering algorithm in data mining. Its objective is to optimize the mean squared error (MSE). The traditional k-means algorithm is not suitable for applications where the sizes of clusters need to be balanced. Given n observations, our objective is to optimize the MSE under the constraint that the observations need to be evenly divided into k clusters. In this paper, we propose an iterative method for the task of clustering with balanced size constraints. Each iteration can be split into two steps, namely an assignment step and an update step. In the assignment step, the data are evenly assigned to each cluster. The balanced assignment task here is formulated as an integer linear program (ILP), and we prove that the constraint matrix of this ILP is totally unimodular. Thus the ILP is relaxed as a linear program (LP) which can be efficiently solved with the simplex algorithm. In the update step, the new centers are updated as the centroids of the observations in the clusters. Assuming that there are n observations and the algorithm needs m iterations to converge, we show that the average time complexity of the proposed algorithm is O ( m n 1 . 65 ) – O ( m n 1 . 70 ) . Experimental results indicate that, comparing with state-of-the-art methods, the proposed algorithm is efficient in deriving more accurate clustering.

Research and application on logistics distribution optimisation problem using big data analysis

The logistics distribution contains wok with large data. To excavate valuable information from the data to improve the reduction and efficiency of logistics distribution cost, the optimisation of logistics distribution centre location is discussed under the environment of big data. The features of logistics distribution and algorithm design idea are provided using basic platform of MapReduce and integrated with data mining clustering algorithm. It is verified to provide a decision scheme for any logistics route optimisation in logistics distribution chain according to the size of the granularity of space division. Compared to traditional clustering, the computation overhead is much less that by Dijkstra distance, The mode using MapReduce to summarise the represents nodes performs parallel computing optimise the efficiency. The scheme also shows practicality and reliability to assist the logistics enterprises with lower cost and better benefit.

RETRACTED ARTICLE: Optimal design of transportation distance in logistics supply chain model based on data mining algorithm

Supply chain management first needs to solve the location problem of distribution center of logistics. At present, the application of the centre-of-gravity method to calculate site selection is relative extensive. Based on the method of extreme distance calculation in the field of data mining technology, a new supply chain distribution center selection method combined with clustering algorithm and the centre-of-gravity selection method was proposed in this paper. In order to avoid the limitation of the traditional centre-of-gravity algorithm, the geographical location price was added to the optimization algorithm as the weight value, and the total cost of the model was calculated; then a “three-segment” data mining clustering algorithm was given to improve the efficiency of clustering calculation and avoid isolation finally, the K-means algorithm, the optimized three-segment algorithm and the hierarchical clustering algorithm and so on were compared, and the simulation calculation was carried out. It can be found that the algorithm of clustering center of gravity of the extreme distance data mining can reduce the cost and is advantageous to solve the problem of the location of the supply chain logistics center of gravity location.