Traditional Mining Algorithms Research Articles

Distributed Approach for Efficiently Extracting Common Patterns from Large Datasets

Abstract: In recent years, advancements in database knowledge discovery have enabled it to find important and practical information with great force. Frequent pattern mining and association rule mining has been thoroughly investigated for a great range of real-world applications. Traditional mining algorithms use memory-resident, centralised data. The performance is not effective enough when applying these methods to remote databases because of the large amount of data, bandwidth limitation, and energy limitations, especially in this era of big data. Datamining in dispersed contexts has therefore become a significant field of study.

Application of Association Rule Mining Algorithm in Talent Training Assessment in Universities Under Social Sustainability

From the contradiction between the difficulty of employment in enterprises and the difficulty of employment in college students, we can see that the strategies of cultivating talents in colleges and universities are biased. This research combines the talent cultivation mode of colleges and universities with the current situation of social development, constructs a comprehensive quality assessment system for students, and uses hierarchical analysis and expert opinions to determine the weight coefficients of each index. In response to the shortcomings of traditional mining algorithms, an improved APRIORI algorithm based on the bit-array matrix is proposed to improve the mining efficiency by introducing the duplicate terms of identification and variable records in data compression. The experimental results show that the improved algorithm can achieve fast convergence on different data sets. The minimum running time of BM Apriori algorithm in different nodes is 102s; under different supports, the average running time of BM Apriori algorithm is 1.49s, and the efficiency is improved by 81.4%. BM Apriori algorithm is used for data mining, and the association rules between courses and students’ grades obtained show that students’ grades are prone to decline during the freshman to sophomore years. Schools should focus on cultivating students’ good learning habits, cultivating excellent talents, and promoting the sustainable development of society.

Design of Rural E-commerce Customer Data Mining System

Customer relationship management based on data mining helps to strengthen the cultivation of high-quality customers and improve customer loyalty and loyalty. China’s e-commerce industry market is becoming more and more prosperous, with the data volume increasing exponentially. Mining the purchasing behavior data of e-commerce industry customers has become a major way for major e-commerce platforms to increase sales volume. Traditional mining algorithms cannot meet the needs of big data analysis. According to different purposes, we will combine mining algorithm and cloud computing platform to mine the potential information of these data and provide beneficial guidance for enterprise operation. This paper starts with the impact of big data on rural e-commerce customers. At the same time, this paper analyzes the current situation of rural e-commerce and proposes the design of rural e-commerce system based on data mining [1].

A Distributed Method for Fast Mining Frequent Patterns From Big Data

In recent years, knowledge discovery in databases provides a powerful capability to discover meaningful and useful information. For numerous real-life applications, frequent pattern mining and association rule mining have been extensively studied. In traditional mining algorithms, data are centralized and memory-resident. As a result of the large amount of data, bandwidth limitation, and energy limitations when applying these methods to distributed databases, especially in this era of big data, the performance is not effective enough. Hence, data mining on distributed environments has emerged as an important research area. To improve the performance, we propose a set of algorithms based on FP growth that discover FPs that are capable of providing fast and scalable service in distributed computing environments and a brief data structure to store items and counts to minimize the data for transmission on the network. To ensure completeness and execution capability, DistEclat and BigFIM were considered for the experiment comparison. Experiments show that the proposed method has superior cost-effectiveness for processing massive datasets and good capabilities under various experiment conditions. The proposed method on average required only 33% of the execution time and 45% of the transmission cost of DistEclat. Compared to BigFIM, The proposed method on average required 23.3% of the execution time and 14.2% of the transmission cost of BigFIM.

Temporal association rules discovery algorithm based on improved index tree

Abstract With the rapid increase of information generated from all kinds of sources, temporal big data mining in business area has been paid more and more attention recently. A novel data mining algorithm for mining temporal association is proposed. Mining temporal association can not only provide better predictability for customer behaviour but also help organisations with better strategies and marketing decisions. To compare the proposed algorithm, two methods to mine temporal association are presented. One is improved based on a traditional mining algorithm, Apriori. The other is based on an Index-Tree. Moreover, the proposed method is extended to mine temporal association in multi-dimensional space. The experimental results show that the Index-Tree method outperforms the Apriori-modified method in all cases.

Big data analytics for retail industry using MapReduce-Apriori framework

Presently, retailing has changed its face from unordered stacked traditional stores to beautifully decorated and appropriately managed merchandise stores or shopping malls with excellent ambiance and comfort. Therefore, these stores try to accommodate all needed items for daily use or rarely required items under the same roof. However, the primary challenge for today’s retailer is that the modern customer is quality and brands conscious as well as compare for services provided to them by different outlets at the comfort of home with a single click. Therefore, customers prefer to purchase from E-Commerce websites instead of physically visiting a retail store, which leads to the downfall in the sales of retailers which become a serious threat to them. Therefore, retailers are required to work sincerely towards their customer expectations by providing all their needed goods under the same roof. Therefore, the objective of this paper is to assist retail business owners to recognize the purchasing needs of their customers and hence to entice customers to physical retail stores away from competitor E-Commerce websites. This paper employs a systematic research methodology based on association rule mining deployed over Map-Reduce based Apriori association mining and Hadoop based intelligent cloud architecture to determine useful buying patterns from purchase history of previous customers, in order to assist retail business owners. The finding acknowledges that the traditional mining algorithms have not progressed to support big data analysis as required by current retail businesses owners. The job of finding unknown association rules from big data requires a lot of resources such as memory and processing engines. Moreover, traditional mining systems are inadequate to provide support for partial failure support, extensibility, scalability etc. Therefore, this study aims to implement and develop MapReduce based Apriori (MR-Apriori) algorithm in the form of Intelligent Retail Mining Tool i.e. IRM Tool to recognize all these concerns in an efficient manner. The proposed system adequately satisfy all significant requisites anticipated from modern Big Data processing systems such as scalability, fault tolerance, partial failure support etc. Finally, this study experimentally verifies the effectiveness of the proposed algorithm i.e. MR-Apriori by speed-up, size-up, and scale-up evaluation parameters.

Apriori semantic processing orienting at the business English parallel corpus characteristics in business contract

English grammar teaching is an indispensable part of language teaching, and learners’ mastery and use of grammatical knowledge have a positive and important role in obtaining speech ability. Aiming at the problem of characteristic processing in business English parallel corpora in business contracts, this paper proposes a similar-Apriori mining semantic processing algorithm under two-stage high expectation weight business English parallel corpus item set. First of all, aiming at the problem that in the traditional hierarchical mining algorithm, only business English parallel corpus item set upper bound is used, which has not ideal effect for the high expectation weight problem, the downward closure characteristics of hierarchy mining algorithm is designed, and the proof procedure is given, this characteristic can effectively reduce the processing quantity of the candidate business English parallel corpus while ensuring the accuracy. Secondly, the two-stage data mining process is constructed, in the first stage, a high expectation weight business English parallel corpus item set is obtained for a set of candidate business English parallel corpus item set based on a hierarchical search method, in the second stage, the database is scanned again to obtain high expectation weight of business English parallel corpus item set, so as to complete the data mining process. Finally, the effectiveness of the algorithm is verified through simulation experiment.

Construction of evaluation system of sports talent training scheme based on data mining

In order to improve the effectiveness of the evaluation system construction for sports talent training program, this paper puts forward a kind of evaluation system construction method for sports talent training program based on data mining. It provides the description target of sports talent training program, the problem that the upper bound of itemsets only used by the traditional mining algorithm has unsatisfactory effect on the high-expectation weight and downward closure property of mining algorithm in design hierarchy, and presents the proof process, and this property can effectively reduce the processing capacity of candidate set in the premise of ensuring the accuracy, to construct the two-stage data mining process. By comparing with the training program of 2007 version, the proposed method can eliminate the setting compulsory course equivalent to individual professional elective course, to promote the diverse development of students.

Finding Abnormal Vessel Trajectories Using Feature Learning

Global Positioning System technology has been widely used in vehicle tracking and road planning applications. An enormous amount of data concerning the trajectories of vehicles has been collected and stored for tracking purposes. A trajectory contains not only the footprints of a moving object but also additional information, such as speed and stopping points. Therefore, the large-scale trajectory data sets provide rich information and are currently attracting considerable attention; there have been many successful studies of event detection based on trajectory data. However, most of these studies have focused only on vehicles traveling in a road network and have note considered maritime trajectories. A maritime trajectory also contains auxiliary data (e.g., speed and rotation) in addition to the movements of a ship. However, ships are not bound to road networks, and consequently, it is difficult to apply traditional mining algorithms based on road networks. In addition, even if the amount of maritime trajectory data is very large, these data are also spatially sparse, which will significantly reduce the effectiveness of most existing mining algorithms. In this paper, we propose a new method of abnormal trajectory detection to address this problem. This method can detect abnormal vessel trajectories from Automatic Identification System (AIS), records for vessels via our feature learning algorithm. To reduce the search space, we invoke reference points as well as the Piecewise Linear Segmentation (PLS), algorithm to compress the trajectories without losing important information. A time-aware and spatially correlated collaborative algorithm is proposed to increase the density of the trajectories to improve the accuracy of the detection algorithm, which is based on Dynamic Time Warping (DTW). Finally, we report experiments conducted on a real-world data set, which demonstrate that the proposed detection method can detect anomalous trajectories effectively.

A Novel Deep Web Data Mining Algorithm based on Multi-Agent Information System and Collaborative Correlation Rule

Along with the rapid advancement of Internet technology and machine learning science, the data mining techniques have been widely applied on the web page information pattern analysis issues. To enhance the traditional mining algorithms theoretically and numerically, we propose the novel deep web data mining algorithm based on multi-agent information system and collaborative correlation rule in this manuscript. Firstly, we review the latest web mining methodologies to serve as the comparison objects. Then, we introduce the revised agent based algorithm. MAS consists of more than one agent, MAS using parallel distributed processing technology and modular design thought and the complex system is divided into relatively independent agent subsystem. Later, we combine the AdaBoost method to propose the collaborative correlation rule. As the combination, we use the mentioned two techniques to form the optimized and enhanced deep web data mining algorithm with the implementation of programming languages. The experimental result proves the feasibility of our approach and compared with other contemporary state-of-the-art algorithms, our method outperforms and achieves better accuracy with low time-consuming.

Weighted FP-Tree Mining Algorithms for Conversion Time Data Flow

The data distribution in the data streams usually changes dynamically with time. Traditional mining algorithms based on transaction are difficult to establish the correlation between time characteristics and relationship features, thus making the results inaccurate. By analyzing the problems in the processing of time data stream, we put forward the concept of time gap degrees and design a mining algorithms based on weighted FP-Tree. We introduce the concept of FP-Tree node weights to transform the time data dynamically and excavate the data stream association rules. The experiments performed on the actual data set show that the algorithm can improve the recall and precision while consumes comparable computational time.

Research on micro-blog based personal interests and emotion transition tendency

With the rapid development of social network,people can present or share their opinions and emotions on the micro-blog,and as a result,large amount of blog texts and emotional information of different topics are produced. As the traditional text mining algorithms cannot deal with these blog texts with fractional contents and different personal emotions effectively,this paper presents a novel weighting computation approach based on the key characters of micro-blog posts to get the dynamic interests of bloggers on different times. Furthermore,based on the emotion classification,the emotion transition tendency analysis can also be implemented. Experimental results show the feasibility of the presented approach.

Implementation of optimum binning, ensemble learning and re-sampling techniques to predict student's performance

Educational data-mining is an emerging area of research that could extract useful information for the students as well as for the instructors. In this research, we explore data mining techniques that predict students' final grade. We validate our method by conducting experiments on data that are related to grade for courses in North South University, the first private university and one of the leading universities in higher education in Bangladesh. We also extend our ideas through discretisation of the continuous attributes by equal width binning and incorporate it on traditional mining algorithms. However, due to imbalanced nature of data, we got lower accuracy for imbalanced classes. We implement two re-sampling techniques, i.e., ROS (random over sampling), RUS (random under sampling). Experimental results show that re-sampling techniques could overcome the problem of imbalanced dataset in classification significantly and improve the performance of the classification models. Moreover, three ensemble techniques, namely, bagging, boosting (AdaBoost) and random forests have been applied in this research to predict the students' academic performance.

Frequent Spatiotemporal Association Patterns Mining Based on Granular Computing

In order to discover multi-dimensional spatiotempor al association patterns, and improve the efficiency of traditional mining algorithms for spatiotemporal association patterns, this paper firstly construct s a star association model based on event, which can sh ow more spatiotemporal information on the basis of the present star association model. Besides traditi onal attributes association with point, line and pl ane, the model also can fully and flexibly express tempo ral association, orientation association, and topol ogy association, namely, it can quickly and simply form multi-dimensional spatiotemporal association patterns. And then for the star association model b ased on event, an algorithm of discovering frequent spatiotemporal association patterns based on granul ar computing is proposed, which is different from traditional association patterns mining algorithms. One is that the algorithm breaks traditional think ing of generating candidate frequent itemsets, namely, it generates candidate frequent itemsets by updatin g the mixed radix numeral. The method is quick and si mple to avoid redundant complicated calculations for adopting complex FP-tree data structure or gene rating candidate by joining frequent itemsets. The other is that the algorithm for discovering frequen t spatiotemporal association patterns only needs to read database once via granular computing, in other words, it discovers each frequent spatiotemporal association pattern via constructing a spatiotempor al information granule, where the intension can be mapped to the mixed radix numeral from the mixed ra dix notation system based on spatiotemporal information system. Finally, this paper further dis cusses the characteristics and the optimal applicat ion environments of the algorithm. Experimental results indicate that the algorithm is simpler and faster than these traditional frequent patterns mining alg orithms on the optimal application environments. Povzetek: Opisana je nova metoda za rudarjenje pros torsko-casovnih vzorcev.

Research of a De-Noising Algorithm Based on Sliding Window

Top-quality and efficient service increases in importance in the telecom service. One of its challenging issues is to deal with the atypical incidents. While the traditional mining algorithms are focus on the high-frequent item sets, a de-noising algorithm related to the atypical incidents still remains unsettled. This paper proposed a de-noising model based on the sliding window. In this model, FP-tree and multi-association rules are introduced to fix the thresholds of the sliding window. Experimental results demonstrate that the proposed algorithm can apply an appropriate data set to the knowledge discovery of the atypical incidents

Personal continuous route pattern mining

In the daily life, people often repeat regular routes in certain periods. In this paper, a mining system is developed to find the continuous route patterns of personal past trips. In order to count the diversity of personal moving status, the mining system employs the adaptive GPS data recording and five data filters to guarantee the clean trips data. The mining system uses a client/server architecture to protect personal privacy and to reduce the computational load. The server conducts the main mining procedure but with insufficient information to recover real personal routes. In order to improve the scalability of sequential pattern mining, a novel pattern mining algorithm, continuous route pattern mining (CRPM), is proposed. This algorithm can tolerate the different disturbances in real routes and extract the frequent patterns. Experimental results based on nine persons’ trips show that CRPM can extract more than two times longer route patterns than the traditional route pattern mining algorithms.

Mining periodic patterns with gap requirement from sequences

We study a problem of mining frequently occurring periodic patterns with a gap requirement from sequences. Given a character sequenceSof lengthLand a patternPof lengthl, we considerPa frequently occurring pattern inSif the probability ofobservingPgiven a randomly picked length-lsubsequence ofSexceeds a certain threshold. In many applications, particularly those related to bioinformatics, interesting patterns areperiodicwith agap requirement. That is to say, the characters inPshould match subsequences ofSin such a way that the matching characters inSare separated by gaps of more or less the same size. We show the complexity of the mining problem and discuss why traditional mining algorithms are computationally infeasible. We propose practical algorithms for solving the problem and study their characteristics. We also present a case study in which we apply our algorithms on some DNA sequences. We discuss some interesting patterns obtained from the case study.

Boosting text segmentation via progressive classification

A novel approach for reconciling tuples stored as free text into an existing attribute schema is proposed. The basic idea is to subject the available text to progressive classification, i.e., a multi-stage classification scheme where, at each intermediate stage, a classifier is learnt that analyzes the textual fragments not reconciled at the end of the previous steps. Classification is accomplished by an ad hoc exploitation of traditional association mining algorithms, and is supported by a data transformation scheme which takes advantage of domain-specific dictionaries/ontologies. A key feature is the capability of progressively enriching the available ontology with the results of the previous stages of classification, thus significantly improving the overall classification accuracy. An extensive experimental evaluation shows the effectiveness of our approach.