Extract Classification Rules Research Articles

Rule extraction from convolutional neural networks for heart disease prediction.

The accurate prediction of heart disease is crucial in the field of medicine. While convolutional neural networks have shown remarkable precision in heart disease prediction, they are often perceived as opaque models due to their complex internal workings. This paper introduces a novel method, named Extraction of Classification Rules from Convolutional Neural Network (ECRCNN), aimed at extracting rules from convolutional neural networks to enhance interpretability in heart disease prediction. The ECRCNN algorithm analyses updated kernels to derive understandable rules from convolutional neural networks, providing valuable insights into the contributing factors of heart disease. The algorithm's performance is assessed using the Statlog (Heart) dataset from the University of California, Irvine's repository. Experimental results underscore the effectiveness of the ECRCNN algorithm in predicting heart disease and extracting meaningful rules. The extracted rules can assist healthcare professionals in making precise diagnoses and formulating targeted treatment plans. In summary, the proposed method bridges the gap between the high accuracy of convolutional neural networks and the interpretability necessary for informed decision-making in heart disease prediction.

Distributed Action-Rule Discovery Based on Attribute Correlation and Vertical Data Partitioning

The paper concerns the problem of action-rule extraction when datasets are large. Such rules can be used to construct a knowledge base in a recommendation system. One of the popular approaches to construct action rules in such cases is to partition the dataset horizontally (personalization) and vertically. Different clustering strategies can be used for this purpose. Action rules extracted from vertical clusters can be combined and used as knowledge discovered from the horizontal clusters of the initial dataset. The number of extracted rules strongly depends on the methods used to complete that task. In this study, we chose a software package called SCARI recently developed by Sikora and his colleagues. It follows a rule-based strategy for action-rule extraction that requires prior extraction of classification rules and generates a relatively small number of rules in comparison to object-based strategies, which discover action rules directly from datasets. Correlation between attributes was used to cluster them. We used an agglomerative strategy to cluster attributes of a dataset and present the results by using a dendrogram. Each level of the dendrogram shows a vertical partition schema for the initial dataset. From all partitions, for each level, action rules are extracted and then concatenated. Their precision, the lightness, and the number of rules are presented and compared. Lightness shows how many action rules can be applied on average for each tuple in a dataset.

Identifying MicroRNA Markers That Predict COVID-19 Severity Using Machine Learning Methods.

Individuals with the SARS-CoV-2 infection may experience a wide range of symptoms, from being asymptomatic to having a mild fever and cough to a severe respiratory impairment that results in death. MicroRNA (miRNA), which plays a role in the antiviral effects of SARS-CoV-2 infection, has the potential to be used as a novel marker to distinguish between patients who have various COVID-19 clinical severities. In the current study, the existing blood expression profiles reported in two previous studies were combined for deep analyses. The final profiles contained 1444 miRNAs in 375 patients from six categories, which were as follows: 30 patients with mild COVID-19 symptoms, 81 patients with moderate COVID-19 symptoms, 30 non-COVID-19 patients with mild symptoms, 137 patients with severe COVID-19 symptoms, 31 non-COVID-19 patients with severe symptoms, and 66 healthy controls. An efficient computational framework containing four feature selection methods (LASSO, LightGBM, MCFS, and mRMR) and four classification algorithms (DT, KNN, RF, and SVM) was designed to screen clinical miRNA markers, and a high-precision RF model with a 0.780 weighted F1 was constructed. Some miRNAs, including miR-24-3p, whose differential expression was discovered in patients with acute lung injury complications brought on by severe COVID-19, and miR-148a-3p, differentially expressed against SARS-CoV-2 structural proteins, were identified, thereby suggesting the effectiveness and accuracy of our framework. Meanwhile, we extracted classification rules based on the DT model for the quantitative representation of the role of miRNA expression in differentiating COVID-19 patients with different severities. The search for novel biomarkers that could predict the severity of the disease could aid in the clinical diagnosis of COVID-19 and in exploring the specific mechanisms of the complications caused by SARS-CoV-2 infection. Moreover, new therapeutic targets for the disease may be found.

Rule extraction using ensemble of neural network ensembles

Neural networks are well-known for their impressive classification performance, and the ensemble learning technique acts as a catalyst to improve this performance even further by integrating multiple networks.However, neural network ensembles, like neural networks, are regarded as a black box because they cannot explain their decision-making process. As a result, despite their high classification performance, neural networks and their ensembles are unsuitable for some applications that require explainable decisions. However, the rule extraction technique can overcome this drawback by representing the knowledge learned by a neural network in the guise of interpretable decision rules. A rule extraction algorithm provides neural networks the ability to justify their classification responses using explainable classification rules. There are several rule extraction algorithms for extracting classification rules from neural networks, but only a few of them use neural network ensembles to generate rules. As a result, this paper proposes a rule extraction algorithm called Rule Extraction Using Ensemble of Neural Network Ensembles (RE-E-NNES) to demonstrate the high performance of neural network ensembles.RE-E-NNES extracts classification rules by ensembling several neural network ensembles. The results demonstrate the efficacy of the proposed RE-E-NNES algorithm in comparison to other existing rule extraction algorithms.

Analysis of Lymphoma-Related Genes with Gene Ontology and Kyoto Encyclopedia of Genes and Genomes Enrichment.

Lymphoma is a serious malignant tumor that contains more than 70 different types and seriously endangers the body's lymphatic system. The lymphatic system is the regulatory center of the immune system and is important in the immune response to foreign antigens and tumors. Studies showed that multiple genetic variants are associated with lymphoma but determining the pathogenic mechanisms remains a challenge. In the present study, we first applied the Gene Ontology (GO) and KEGG pathway enrichment analyses of lymphoma-associated and lymphoma-nonassociated genes. Next, the Boruta and max-relevance and min-redundancy feature selection methods were performed to filter and rank features. Then, features preselected and ranked using the incremental feature selection method were applied for the decision tree model to identify the best GO terms and KEGG pathways and extract classification rules. Results indicate that our predicted features, such as B-cell activation, negative regulation of protein processing, negative regulation of mast cell cytokine production, and natural killer cell-mediated cytotoxicity, are associated with the biological process of lymphoma, consistent with those of recent publications. This study provides a new perspective for future research on the molecular mechanisms of lymphoma.

Data Mining Approach in Diagnosis and Treatment of Chronic Kidney Disease

Abstract Chronic kidney disease is a general definition of kidney dysfunction that lasts more than 3 months. When chronic kidney disease is advanced, the kidneys are no longer able to cleanse the blood of toxins and harmful waste products and can no longer support the proper function of other organs. The disease can begin suddenly or develop latently over a long period of time without the presence of characteristic symptoms. The most common causes are other chronic diseases – diabetes and hypertension. Therefore, it is very important to diagnose the disease in early stages and opt for a suitable treatment - medication, diet and exercises to reduce its side effects. The purpose of this paper is to analyse and select those patient characteristics that may influence the prevalence of chronic kidney disease, as well as to extract classification rules and action rules that can be useful to medical professionals to efficiently and accurately diagnose patients with kidney chronic disease. The first step of the study was feature selection and evaluation of its effect on classification results. The study was repeated for four models – containing all available patient data, containing features identified by doctors as major factors in chronic kidney disease, and models containing features selected using Correlation Based Feature Selection and Chi-Square Test. Sequential Minimal Optimization and Multilayer Perceptron had the best performance for all four cases, with an average accuracy of 98.31% for SMO and 98.06% for Multilayer Perceptron, results that were confirmed by taking into consideration the F1-Score, for both algorithms was above 0.98. For all these models the classification rules are extracted. The final step was action rule extraction. The paper shows that appropriate data analysis allows for building models that can support doctors in diagnosing a disease and support their decisions on treatment. Action rules can be important guidelines for the doctors. They can reassure the doctor in his diagnosis or indicate new, previously unseen ways to cure the patient.

Classification and action rules in identification and self-care assessment problems.

Disability, especially in children, is a very important and current problem. Lack of proper diagnosis and care increases the difficulty for children to adapt to disabilities. Disabled children have many problems with basic activities of daily living. Therefore, it is very important to support diagnosticians and physiotherapists in recognizing self-care problems in children. The aim of this paper is to extract classification and action rules, useful for those who work with children with disabilities. First, features and their impact on the accuracy of classification are determined. Then, two models are built: one with all features and one with selected ones. For these models the classification rules are extracted. Finally, action rules are mined and the next step in treatment process is predicted. Seventeen features with the greatest impact on classifying a child into a particular group of self-care problems were identified. Based on the implemented algorithms, decision and action rules were obtained. The obtained model, selected attributes and extracted classification and action rules can support the work of therapists and direct their work to those areas of disability where even a minimal reduction of features would be of great benefit to the children.

A knowledge discovery and visualisation method for unearthing emotional states from physiological data

In this paper we propose a knowledge discovery and visualisation method for unearthing emotional states from physiological data typically available from wearable devices. In addition we investigate the viability of using a limited set of wearable sensors to extract decision tree rules which are representative of physiological changes taking place during emotional changes. Our method utilised a fusion of pre-processing and classification techniques using decision trees to discover logic rules relating to the valence and arousal emotional dimensions. This approach normalised the signal data in a manner that enabled accurate classification and generated logic rules for knowledge discovery. Furthermore, the use of three target classes for the emotional dimensions was effective at denoising the data and further enhancing classification and useful rule extraction. There are three key contributions in this work, firstly an exploration and validation of our knowledge discovery methodology, secondly successful extraction of high accuracy rules derived from physiological data and thirdly knowledge discovery and visualisation of relationships within-participant physiological data that can be inferred relating to emotions. Additionally, this work may be utilised in areas such as the medical sciences where interpretable rules are required for knowledge discovery.

Extracting Classification Rules from Artificial Neural Network Trained with Discretized Inputs

Rule extraction from artificial neural networks remains important task in complex diseases such as diabetes and breast cancer where the rules should be accurate and comprehensible. The quality of rules is improved by the improvement of the network classification accuracy which is done by the discretization of input attributes. In this paper, we developed a rule extraction algorithm based on multiobjective genetic algorithms and association rules mining to extract highly accurate and comprehensible classification rules from ANN’s that have been trained using the discretization of the continuous attributes. The data pre-processing provides very good improvement of the ANN accuracy and consequently leads to improve the performance of the classification rules in terms of fidelity and coverage. The results show that our algorithm is very suitable for medical decision making, so an excellent average accuracy of 94.73 has been achieved for the Pima dataset and 99.36 for the breast cancer dataset.

A parallel fuzzy rule-base based decision tree in the framework of map-reduce

Decision trees are commonly used for learning and extracting classification rules from data. The fuzzy rule based decision tree (FRDT) is very representative owing to its better robustness and generalization. However, FRDT cannot work well on the analysis of large-scale data sets. One solution for this problem is parallel computing. A proved effective parallel computing model is Map-Reduce. Ensemble learning is an effective strategy which can significantly improve the generalization ability of machine learning systems. The objective of this paper is to develop a fuzzy rule-base based decision tree on the strategies of parallel computing and ensemble learning. First, we implement a parallel fusing fuzzy rule based classification system via Map-Reduce (MR-FFRCS) to display how to extract fuzzy rules from data in parallel and how to evaluate the fuzzy rules in an ensemble learning way. Then, taking MR-FFRCS as a fundamental module, we propose a parallel fuzzy rule-base based decision tree (MR-FRBDT) to improve the original FRDT algorithm. The experimental studies mainly focus on feasibility and parallelism. Compared with FRDT on 23 UCI benchmark data sets, the proposed MR-FRBDT algorithm with fewer parameters is effective and has the ability to handle large-scale data sets. Furthermore, some numerical experiments conducted on several large-scale data sets verify the parallel performance on reducing computing time and avoiding memory restrictions.

Generalization Driven Fuzzy Classification Rules Extraction using OLAM Data Cubes

An fuzzy classification rules extraction model for online analytical mining (OLAM) was explained in this article. The efficient integration of the concept of data warehousing, online analytical processing (OLAP) and data mining systems converges to OLAM results in an efficient decision support system. Even after associative classification proved as most efficient classification technique there is a lack of associative classification proposals in field of OLAM. While most of existing data cube models claims their superiority over other the fuzzy multidimensional data cubes proved to be more intuitive in user perspective and effectively manage data imprecision. Considering these factors, in this paper we propose an associative classification model which can perform classification over fuzzy data cubes. Our method aimed to improve accuracy and intuitive ness of classification model using fuzzy concepts and hierarchical relations. We also proposed a generalization-based criterion for ranking associative classification rules to improve classifier accuracy. The model accuracy tested on UCI standard database.

Ant Miner

In data mining the task of extracting classification rules from large data is an important task and is gaining considerable attention. This article presents a novel ant miner for classification rule mining. The ant miner is inspired by researches on the behaviour of real ant colonies, simulated annealing, and some data mining concepts as well as principles. This paper presents a Pittsburgh style approach for single objective classification rule mining. The algorithm is tested on a few benchmark datasets drawn from UCI repository. The experimental outcomes confirm that ant miner-HPB (Hybrid Pittsburgh Style Classification) is significantly better than ant-miner-PB (Pittsburgh Style Classification).

Rough Sets Base Associative Classification Rules Extraction from Big Data

Big Data is a current burning challenge for the data analytics research community. Many conventional data analytics techniques have been extended to the MapReduce framework to process Big Data. But in our literature review, we find that for the MapReduce system there is an absolute lack of rough setbased technique. To facilitate this and recognize the importance of the rule-based classification techniques, we suggest a roughset associative classification rules extraction process for the MapReduce framework. The implementation and evaluation of the Big Data Standard data set demonstrated the efficiency of our suggested approach.

A new rule-based knowledge extraction approach for imbalanced datasets

Classification consists of extracting a classifier from large datasets. A dataset is imbalanced if it contains more instances in one class compared to the others. An imbalanced dataset contains majority instances and minority ones. It is worth noting that classical learning algorithms have a bias toward majority instances. If classification is applied to imbalanced datasets, it is called partial classification. Its approaches are generally based on sampling methods or algorithmic methods. In this paper, we propose a new hybrid approach using a three-phase-rule-based extraction process. Initially, the first classifier is extracted; it contains classification rules representing only majority instances. Then, we delete the majority instances, which are well classified by these rules, to produce a balanced dataset. The deleted majority instances are replaced by the extracted classification rules, which prevent any information loss. Subsequently, our algorithm is applied to the obtained balanced dataset to produce the second classifier which contains rules that represent both majority and minority instances. Finally, we add the rules of the first classifier to the second classifier to obtain the final classifier, which will be post-processed. Our approach has been tested on several imbalanced binary datasets. The obtained results show its efficiency compared to other results.

The Building Extraction Based on Object Oriented Classification Method in High Vegetation Coverage Area

Quickly extraction of building information technology is an important application in urban development planning, electronic information, national defense and others. This paper takes Landsat-8 multispectral and panchromatic data as data source, using the local variance method to select the optimal segmentation scale, normalized difference vegetation index (NDVI) and the normalized building index (NDBI) and panchromatic brightness value of an object oriented classification rule extraction. The high vegetation coverage area of buildings, and through the spatial relationships and distinguishing feature of collections of buildings independent buildings and villages. The results showed that Google earth high resolution image analysis and accuracy evaluation. the results of the extraction based on the overall accuracy of village extraction was 83%, the accuracy of extraction of independent buildings was 70%, according to the L8 remote sensing data, object oriented classification method can quickly and accurately extract the high vegetation coverage area of the building.

A Method of Deep Belief Network Image Classification Based on Probability Measure Rough Set Theory

The increasing diverse demand for image feature recognition and complicated relationships among image pixels cannot be fully and effectively handled by traditional single image recognition methods. In order to effectively improve classification accuracy in image processing, a deep belief network (DBN) classification model based on probability measure rough set theory is proposed in our research. First, the incomplete and inaccurate fuzzy information in the original image is preprocessed by the rough set method based on probability measure. Second, the attribute features of the image information are extracted, the attribute feature set is reduced to generate the classification rules, and key components are extracted as the input of the DBN. Third, the network structure of the DBN is determined by the extracted classification rules, and the importance of the rough set attributes is integrated and the weights of the neuronal nodes are corrected by the backpropagation (BP) algorithm. Last, the DBN is trained to classify images. The experimental analysis of the proposed method for medical imagery shows that it is more effective than current single rough set approach or the taxonomy of deep learning.

Generating Classification Rules from Training Samples

In this paper, we describe an algorithm to extract classification rules from training samples using fuzzy membership functions. The algorithm includes steps for generating classification rules, eliminating duplicate and conflicting rules, and ranking extracted rules. We have developed software to implement the algorithm using MATLAB scripts. As an illustration, we have used the algorithm to classify pixels in two multispectral images representing areas in New Orleans and Alaska. For each scene, we randomly selected 10 per cent of the samples from our training set data for generating an optimized rule set and used the remaining 90 per cent of samples to validate the extracted rules. To validate extracted rules, we built a fuzzy inference system (FIS) using the extracted rules as a rule base and classified samples from the training set data. The results in terms of confusion matrices are presented in the paper.

Extended fuzzy hyperline-segment neural network with classification rule extraction

The hyperline segment layer of fuzzy hyperline segment neural network (FHLSNN) consists of number of n- dimensional hyperline segments. Each hyperline segment has the two end points defined in terms of continuous attribute values and an associated membership function. The membership function processes the continuous attribute values and gives the membership of each input pattern to the hyperline segments. These end points and membership function are defined only in terms of continuous attributes and cannot process discrete attributes. But the real world data comprises both continuous and discrete attributes. Due to this original FHLSNN and its ancestor versions cannot be applied to these types of data. Also, justification of the classification decision given by these fuzzy min–max neural networks is required to be obtained to make them more applicable to the real world applications. The objective of the proposed extended fuzzy hyperline segment neural network (EFHLSNN) is to solve these two problems. In the EFHLSNN, each hyperline segment has two end points defined not only in terms of continuous attributes but also the set of binary strings defined for discrete attributes. The membership function and expansion condition of the original FHLSNN are modified to process both continuous and discrete attribute values. The proposed EFHLSNN model is applied to eight different benchmark datasets. The experimental results show that the proposed model gives very good accuracy and the compact rule set that justifies the classification decision of the EFHLSNN and thus, giving rise to good rule fidelity.

Extraction and optimization of classification rules for temporal sequences: Application to hospital data

This study focuses on the problem of supervised classification on heterogeneous temporal data featuring a mixture of attribute types (numeric, binary, symbolic, temporal). We present a model for classification rules designed to use both non-temporal attributes and sequences of temporal events as predicates. We also propose an efficient local search-based metaheuristic algorithm to mine such rules in large scale, real-life data sets extracted from a hospital’s information system. The proposed algorithm, MOSC (Multi-Objective Sequence Classifier), is compared to standard classifiers and previous works on these real data sets and exhibits noticeably better classification performance. While designed with medical applications in mind, the proposed approach is generic and can be used for problems from other application domains.

Accurate and interpretable classification of microspectroscopy pixels using artificial neural networks.

This paper addresses the problem of classifying materials from microspectroscopy at a pixel level. The challenges lie in identifying discriminatory spectral features and obtaining accurate and interpretable models relating spectra and class labels. We approach the problem by designing a supervised classifier from a tandem of Artificial Neural Network (ANN) models that identify relevant features in raw spectra and achieve high classification accuracy. The tandem of ANN models is meshed with classification rule extraction methods to lower the model complexity and to achieve interpretability of the resulting model. The contribution of the work is in designing each ANN model based on the microspectroscopy hypothesis about a discriminatory feature of a certain target class being composed of a linear combination of spectra. The novelty lies in meshing ANN and decision rule models into a tandem configuration to achieve accurate and interpretable classification results. The proposed method was evaluated using a set of broadband coherent anti-Stokes Raman scattering (BCARS) microscopy cell images (600 000 pixel-level spectra) and a reference four-class rule-based model previously created by biochemical experts. The generated classification rule-based model was on average 85% accurate measured by the DICE pixel label similarity metric, and on average 96% similar to the reference rules measured by the vector cosine metric.