Step In Data Mining Research Articles

An Extended Laplacian Score Algorithm for Unsupervised Feature Selection

Experts from various sectors, utilize data mining techniques to discover most useful information from the huge amount of data, to improve their quality of outcomes. The Presence of irrelevant and redundant features affects the accuracy of mining result. Before applying any mining technique, the data need to be preprocessed. Feature selection, a preprocessing step in data mining provides better mining performance. In this paper, we propose a new two step algorithm for unsupervised feature selection. In the first step Laplacian Score is used to select the important features. And in the second step, Symmetric Uncertainty is used to remove redundant features. The experimental results show that the proposed algorithm outperforms the Laplacian Score algorithm.

The optimal combination of feature selection and data discretization: An empirical study

Feature selection and data discretization are two important data pre-processing steps in data mining, with the focus in the former being on filtering out unrepresentative features and in the latter on transferring continuous attributes into discrete ones. In the literature, these two domain problems have often been studied, individually. However, the combination of these two steps has not been fully explored, although both feature selection and discretization may be required for some real-world datasets. In this paper, two different combination orders of feature selection and discretization are examined in terms of their classification accuracies and computational times. Specifically, filter, wrapper, and embedded feature selection methods are employed, which are PCA, GA, and C4.5, respectively. For discretization, both supervised and unsupervised learning based discretizers are used, specifically MDLP, ChiMerge, equal frequency binning, and equal width binning. The experimental results, based on 10 UCI datasets, show that, for the SVM classifier performing MDLP first and C4.5 second outperforms the other combinations. Not only is less computational time required but this also provides the highest rate of classification accuracy. For the decision tree classifier, performing C4.5 first and MDLP second is recommended.

A novel hybrid feature selection strategy in quantitative analysis of laser-induced breakdown spectroscopy

Laser-induced breakdown spectroscopy (LIBS) has been recognized as a significant tool for quantitative analysis of elements with its unique advantages, especially in combination with multivariate calibration methods. However, LIBS spectra typically include large amounts of high-dimensional data that affect the predictive accuracy of multivariate calibration based on LIBS. Feature selection, as an important preprocessing step in data mining, can improve the performance of the multivariate calibration model by eliminating the redundant and irrelevant features. In this study, a hybrid feature selection method based on V-WSP-PSO was proposed to improve the accuracy of LIBS analysis. The proposed method combines the advantages of V-WSP based filter method and particle swarm optimization (PSO) based wrapper method. The uncorrelated and redundant features were first eliminated by V-WSP method to form a simplified input subset, and then the retained features were further refined by PSO method to find a small set of features with high predictive accuracy. In order to evaluate the performance of proposed method, LIBS experiments were performed using 28 coal samples, and a nonlinear multivariate calibration method based on kernel extreme learning machine (KELM) was selected to implement the proposed hybrid feature selection method for calorific value determination of coal. Comparing the proposed method with several other feature selection methods shows that the V-WSP-PSO method is best in terms of number of selected features and predictive accuracy. Finally, 114 features were selected from full spectrum (27620 features) by V-WSP-PSO method and the best root mean square error of cross validation (RMSECV) and determination coefficient of cross validation (RCV2) were 0.4013 MJ/kg and 0.9908, root mean square error of prediction (RMSEP) and determination coefficient of prediction (RP2) were 0.3534 MJ/kg and 0.9894. The overall results demonstrate that the V-WSP-PSO method is more efficient to reduce the redundant features, calculation time and improve the model performance, and it is a good alternative for feature selection in multivariate calibration.

M1CR0B1AL1Z3R-a user-friendly web server for the analysis of large-scale microbial genomics data.

Large-scale mining and analysis of bacterial datasets contribute to the comprehensive characterization of complex microbial dynamics within a microbiome and among different bacterial strains, e.g., during disease outbreaks. The study of large-scale bacterial evolutionary dynamics poses many challenges. These include data-mining steps, such as gene annotation, ortholog detection, sequence alignment and phylogeny reconstruction. These steps require the use of multiple bioinformatics tools and ad-hoc programming scripts, making the entire process cumbersome, tedious and error-prone due to manual handling. This motivated us to develop the M1CR0B1AL1Z3R web server, a ‘one-stop shop’ for conducting microbial genomics data analyses via a simple graphical user interface. Some of the features implemented in M1CR0B1AL1Z3R are: (i) extracting putative open reading frames and comparative genomics analysis of gene content; (ii) extracting orthologous sets and analyzing their size distribution; (iii) analyzing gene presence–absence patterns; (iv) reconstructing a phylogenetic tree based on the extracted orthologous set; (v) inferring GC-content variation among lineages. M1CR0B1AL1Z3R facilitates the mining and analysis of dozens of bacterial genomes using advanced techniques, with the click of a button. M1CR0B1AL1Z3R is freely available at https://microbializer.tau.ac.il/.

Hybrid Efficient Genetic Algorithm for Big Data Feature Selection Problems

Due to the huge amount of data being generating from different sources, the analyzing and extracting of useful information from these data becomes a very complex task. The difficulty of dealing with big data optimization problems comes from many factors such as the high number of features, and the existing of lost data. The feature selection process becomes an important step in many data mining and machine learning algorithms to reduce the dimensionality of the optimization problems and increase the performance of the classification or clustering algorithms. In this paper, a set of hybrid and efficient genetic algorithms are proposed to solve feature selection problem, when the handled data has a large feature size. The proposed algorithms use a new gene-weighted mechanism that can adaptively classify the features into strong relative features, weak or redundant features, and unstable features during the evolution of the algorithm. Based on this classification, the proposed algorithm gives the strong features high priority and the weak features less priority when generating new candidate solutions. In the same time, the proposed algorithm tries to more concentrate on unstable features that sometimes appear and sometimes disappear from the best solutions of the population. The performance of proposed algorithms is investigated by using different datasets and feature selection algorithms. The results show that our proposed algorithms can outperform the other feature selection algorithms and effectively enhance the classification performance over the tested datasets.

Preprocessing of Alarm Data for Data Mining

Many industries, including process industry, face an increased number of alarms every day. This is due to advanced computer-based monitoring and control technologies that are widely available in all industrial plants. On the other hand, data mining, as a method of finding patterns in data, has been widely used to discover patterns and relationships in alarm data, in hopes of reducing the volume of alarms and operators’ workload. One of the first steps in data mining is to prepare and cleanup raw data for better mining, also known as preprocessing. In this paper, we focus on preprocessing of alarm data and investigate the steps required for data preparation. Two steps—namely, removing chattering alarms and reconstruction of missing alarms—are more challenging. For chattering alarms, many algorithms are proposed with a discussion on the time frame that should be selected for removing chattering alarms. As for the reconstruction of missing alarms, two methods are presented, using information from the same alarm tag or other related alarms. A case study shows the efficiency of the proposed methods.

An Attribute Reduction Method Using Neighborhood Entropy Measures in Neighborhood Rough Sets.

Attribute reduction as an important preprocessing step for data mining, and has become a hot research topic in rough set theory. Neighborhood rough set theory can overcome the shortcoming that classical rough set theory may lose some useful information in the process of discretization for continuous-valued data sets. In this paper, to improve the classification performance of complex data, a novel attribute reduction method using neighborhood entropy measures, combining algebra view with information view, in neighborhood rough sets is proposed, which has the ability of dealing with continuous data whilst maintaining the classification information of original attributes. First, to efficiently analyze the uncertainty of knowledge in neighborhood rough sets, by combining neighborhood approximate precision with neighborhood entropy, a new average neighborhood entropy, based on the strong complementarity between the algebra definition of attribute significance and the definition of information view, is presented. Then, a concept of decision neighborhood entropy is investigated for handling the uncertainty and noisiness of neighborhood decision systems, which integrates the credibility degree with the coverage degree of neighborhood decision systems to fully reflect the decision ability of attributes. Moreover, some of their properties are derived and the relationships among these measures are established, which helps to understand the essence of knowledge content and the uncertainty of neighborhood decision systems. Finally, a heuristic attribute reduction algorithm is proposed to improve the classification performance of complex data sets. The experimental results under an instance and several public data sets demonstrate that the proposed method is very effective for selecting the most relevant attributes with great classification performance.

Feature selection based on community detection in feature correlation networks

Feature selection is an important data preprocessing step in data mining and machine learning tasks, especially in the case of high dimensional data. In this paper, we propose a novel feature selection method based on feature correlation networks, i.e. complex weighted networks describing the strongest correlations among features in a dataset. The method utilizes community detection techniques to identify cohesive groups of features in feature correlation networks. A subset of features exhibiting a strong association with the class variable is selected according to the identified community structure taking into account the size of feature communities and connections within them. The proposed method is experimentally evaluated on a high dimensional dataset containing signaling protein features related to the diagnosis of Alzheimer’s disease. We compared the performance of seven commonly used classifiers that were trained without feature selection, after feature selection by four variants of our method determined by different community detection techniques, and after feature selection by four widely used state-of-the-art feature selection methods available in the WEKA machine learning library. The results of the experimental evaluation indicate that our method improves the classification accuracy of several classification models while greatly reducing the dimensionality of the dataset. Additionally, our method tends to outperform traditional feature selection methods provided by the WEKA library.

Aplicação de mineração de dados em informações oriundas de prontuários de paciente

Este artigo procura investigar a aplicação da Mineração de Dados na descoberta de conhecimento oriundo de informações provenientes de prontuários do paciente. Diante disso, o objetivo foi examinar a bibliografia na busca da utilização, resultados e investimentos na área. A metodologia utilizada consistiu no levantamento bibliográfico, por meio de revisão de literatura e a aplicação de uma etapa da mineração de dados, a importação em dados provenientes da saúde. Conclui-se que a Mineração de Dados é eficiente, já existem muitas pesquisas e investimentos de grandes empresas e neste momento, possui um grande potencial de crescimento.

A Novel, Fast, Reliable, and Data-Driven Method for Simultaneous Single-Trial Mining and Amplitude-Latency Estimation Based on Proximity Graphs and Network Analysis.

Both amplitude and latency of single-trial EEG/MEG recordings provide valuable information regarding functionality of the human brain. In this article, we provided a data-driven graph and network-based framework for mining information from multi-trial event-related brain recordings. In the first part, we provide the general outline of the proposed methodological approach. In the second part, we provide a more detailed illustration, and present the obtained results on every step of the algorithmic procedure. To justify the proposed framework instead of presenting the analytic data mining and graph-based steps, we address the problem of response variability, a prerequisite to reliable estimates for both the amplitude and latency on specific N/P components linked to the nature of the stimuli. The major question addressed in this study is the selection of representative single-trials with the aim of uncovering a less noisey averaged waveform elicited from the stimuli. This graph and network-based algorithmic procedure increases the signal-to-noise (SNR) of the brain response, a key pre-processing step to reveal significant and reliable amplitude and latency at a specific time after the onset of the stimulus and with the right polarity (N or P). We demonstrated the whole approach using electroencephalography (EEG) auditory mismatch negativity (MMN) recordings from 42 young healthy controls. The method is novel, fast and data-driven succeeding first to reveal the true waveform elicited by MMN on different conditions (frequency, intensity, duration, etc.). The proposed graph-oriented algorithmic pipeline increased the SNR of the characteristic waveforms and the reliability of amplitude and latency within the adopted cohort. We also demonstrated how different EEG reference schemes (REST vs. average) can influence amplitude-latency estimation. Simulation results revealed robust amplitude-latency estimations under different SNR and amplitude-latency variations with the proposed algorithm.

Feature selection based on artificial bee colony and gradient boosting decision tree

Data from many real-world applications can be high dimensional and features of such data are usually highly redundant. Identifying informative features has become an important step for data mining to not only circumvent the curse of dimensionality but to reduce the amount of data for processing. In this paper, we propose a novel feature selection method based on bee colony and gradient boosting decision tree aiming at addressing problems such as efficiency and informative quality of the selected features. Our method achieves global optimization of the inputs of the decision tree using the bee colony algorithm to identify the informative features. The method initializes the feature space spanned by the dataset. Less relevant features are suppressed according to the information they contribute to the decision making using an artificial bee colony algorithm. Experiments are conducted with two breast cancer datasets and six datasets from the public data repository. Experimental results demonstrate that the proposed method effectively reduces the dimensions of the dataset and achieves superior classification accuracy using the selected features.

Distributed correlation-based feature selection in spark

Feature selection (FS) is a key preprocessing step in data mining. CFS (Correlation-Based Feature Selection) is an FS algorithm that has been successfully applied to classification problems in many domains. We describe Distributed CFS (DiCFS) as a completely redesigned, scalable, parallel and distributed version of the CFS algorithm, capable of dealing with the large volumes of data typical of big data applications. Two versions of the algorithm were implemented and compared using the Apache Spark cluster computing model, currently gaining popularity due to its much faster processing times than Hadoop’s MapReduce model. We tested our algorithms on four publicly available datasets, each consisting of a large number of instances and two also consisting of a large number of features. The results show that our algorithms were superior in terms of both time-efficiency and scalability. In leveraging a computer cluster, they were able to handle larger datasets than the non-distributed WEKA version while maintaining the quality of the results, i.e., exactly the same features were returned by our algorithms when compared to the original algorithm available in WEKA.

Cost-sensitive feature selection via the ℓ2,1-norm

An essential step in data mining and machine learning is selecting a useful feature subset from the high-dimensional feature space. Many existing feature selection algorithms only consider precision, but do not consider error types and test cost. In this paper, we use the ℓ2,1-norm to propose a cost-sensitive embedded feature selection algorithm that minimizes the total cost rather than maximizing accuracy. The algorithm is a cost-sensitive feature selection algorithm with joint ℓ2,1-norm minimization of the loss function with misclassification costs. The ℓ2,1-norm based loss function with misclassification costs is robust to outliers. We also add an orthogonal constraint term to guarantee that each selected feature is independent. The proposed algorithm simultaneously takes into account both test costs and misclassification costs. Finally, an iterative updating algorithm is provided using the objective function that makes cost-sensitive feature selection more efficient. The cost-sensitive feature selection algorithm is more realistic than existing feature selection algorithms. Extensive experimental results on publicly available datasets demonstrate that the proposed algorithm is effective, can select a low-cost subset and achieve better performance than other feature selection algorithms in real-world applications.

Granular maximum decision entropy-based monotonic uncertainty measure for attribute reduction

Attribute reduction is considered an important preprocessing step in machine learning, pattern recognition, and data mining, and several attribute reduction measures based on rough set theory have been introduced to deal with vague, imprecise, or uncertain data. However, some of the measures inherently suffer from nonmonotonicity and redundancy. In this paper, a monotonic uncertainty measure, called granular maximum decision entropy (GMDE), is proposed. Specifically, we first develop a notion of maximum decision entropy. By integrating the uncertainty of the maximum decision entropy with the granulation of knowledge, a novel uncertainty measure is then presented, and its monotonicity is theoretically proved. We also provide a forward heuristic attribute reduction algorithm based on the proposed uncertainty measure, which could simultaneously select the informative attributes and remove the unnecessary attributes in the procedure of attribute reduction, thus resulting in high efficiency. The experiments conducted on several UCI data sets demonstrate that the proposed measure and algorithm are effective and computationally inexpensive and are superior to the representatives in terms of classification performance and efficiency.

Contributions of KDD to the Knowledge Management Process


 
 
 Knowledge Discovery in Databases (KDD), as any organizational process, is carried out beneath a Knowledge Management (KM) model adopted (even informally) by a corporation. KDD is grossly described in three steps: pre-processing, data mining, and post-processing. The latter is mainly related to the task of transforming in knowledge the patterns issued in the data mining step. On the other hand, KM comprises the following phases, in which knowledge is the subject of the actions: identification of abilities, acquisition, selection and validation, organization and storage, sharing, application, and creation. Although there are many overlaps between KDD and KM, one of them is broadly recognized: the point in which knowledge arises. This paper concerns a study aimed at clarifying relations between the overlapping areas of KDD and knowledge creation, in KM. The work is conducted by means of a case study using the data from the Electoral Court of the Federal District (ECFD), Brazil. The study was developed over a 1.717.000-citizens data set from which data mining models were built by applying algorithms from Weka. It was observed that, although the importance of Information Technology is well recognized in the KM realm, the techniques of KDD deserve a special place in the knowledge creation phase of KM. Moreover, beyond the overlap of post- processing and knowledge creation, other steps of KDD can contribute significantly to KM. An example is the fact that one important decision taken from the ECFD board was taken on the basis of a knowledge acquired from the pre-processing step of KDD.
 
 

An incremental attribute reduction method for dynamic data mining

As an important preprocessing step for data mining, attribute reduction has become a hot research topic in rough set theory. In practice, many real data may vary dynamically with time, therefore, reduct will change dynamically under the variation of objects and attributes in decision systems. The classical attribute reduction methods need to recompute from scratch, which are ineffective to deal with dynamic decision systems. How to implement updating reducts by utilizing previous results is vital for improving the efficiency of attribute reduction approaches. In the paper, we firstly introduce incremental mechanisms of computing reduct when objects and attributes of the decision system change dynamically. Then, incremental methods are developed to update reduct when attributes and objects increase simultaneously. Finally, a series of experiments are conducted to validate the proposed incremental attribute reduction methods. Experimental results show that they are effective to update reduct with change of attributes and objects in the decision systems.

Rough sets and Laplacian score based cost-sensitive feature selection

Cost-sensitive feature selection learning is an important preprocessing step in machine learning and data mining. Recently, most existing cost-sensitive feature selection algorithms are heuristic algorithms, which evaluate the importance of each feature individually and select features one by one. Obviously, these algorithms do not consider the relationship among features. In this paper, we propose a new algorithm for minimal cost feature selection called the rough sets and Laplacian score based cost-sensitive feature selection. The importance of each feature is evaluated by both rough sets and Laplacian score. Compared with heuristic algorithms, the proposed algorithm takes into consideration the relationship among features with locality preservation of Laplacian score. We select a feature subset with maximal feature importance and minimal cost when cost is undertaken in parallel, where the cost is given by three different distributions to simulate different applications. Different from existing cost-sensitive feature selection algorithms, our algorithm simultaneously selects out a predetermined number of “good” features. Extensive experimental results show that the approach is efficient and able to effectively obtain the minimum cost subset. In addition, the results of our method are more promising than the results of other cost-sensitive feature selection algorithms.

A VNS algorithm for feature selection in hierarchical classification context

Feature selection, usually adopted as a preprocessing step for data mining, is used to select a subset of predictive features aiming to improve the performance of a predictive model. Despite of the benefits of feature selection for classification task, to the best of our knowledge, there is no work in the literature that addresses feature selection in conjunction with global hierarchical classifiers. Thus, in this paper, we fill this gap proposing a feature selection method based on Variable Neighborhood Search (VNS) metaheuristic for the hierarchical classification context. Computational experiments were carried out on five bioinformatics datasets to evaluate the effect of the proposed algorithm on classification performance when using a global hierarchical classifier. As result, we have obtained a classifier performance improvement for three datasets and a competitive result for a fourth dataset, which indicates the suitability of the proposed method for the hierarchical classification scenario.

OBJECT-BASED ANALYSIS FOR URBAN LAND COVER MAPPING USING THE INTERIMAGE AND THE SIPINA FREE SOFTWARE PACKAGES

Abstract: In this work we introduce an object-based method, applied to urban land cover mapping. The method is implemented with two open-source tools: SIPINA, a data mining software package; and InterIMAGE, an object-based image analysis system. Initially, segmentation, feature extraction and sample selection procedures are performed with InterIMAGE. In order to reduce the time and subjectivity involved to develop the decision rules in InterIMAGE, a data mining step is then carried out with SIPINA. In sequence, the decision trees delivered by SIPINA are analysed and encoded into InterIMAGE decision rules for the final classification step. Experiments were conducted using a subset of a GeoEye image, acquired in January 01, 2013, covering the urban portion of the municipality of Goianésia, Brazil. Five decision tree induction algorithms, available in SIPINA, were tested: ID3, C45, GID3, Assistant86 and CHAID. The TAU and Kappa coefficients were used to evaluate the results. The TAU values obtained were in the range of 0.66 and 0.70, while those for Kappa varied from 0.65 to 0.69.

Optimization Research of Decision Support System Based on Data Mining Algorithm

In order to analyze and make use of data and information more accurately and efficiently, the clustering algorithm is further studied to a certain extent, especially in the process of clustering, further analyzes and refines the processing data. In the field of application of previous clustering of mathematical statistics such as pattern recognition, do some reference, especially in the ant colony clustering algorithm in data aggregation is proposed based on the principle of solving business decision support system to deal with data in the huge data processing result is not ideal problem. The data processing steps of data mining are studied. Through information entropy and ant colony clustering algorithm to achieve this process, the original data for reasoning and verification is also used and the effect before and after the improvement are compared. At the same time, this research provides effective decision support for website construction in contemporary e-commerce field. Taking e-commerce website browsing path as an example, applying the idea of ant colony clustering algorithm based on information entropy to carry out path analysis, five kinds of path types are got. This study can be used as a reference for the construction of other e-commerce websites.