Set Of Classification Rules Research Articles

A New Classification Rule-Set for Mapping Surface Water in Complex Topographical Regions Using Sentinel-2 Imagery

Surface water is a critical natural resource, but its mapping accuracy is vulnerable to cloud cover, snow, shadows, and diverse roofing materials. Recognizing the limitations of a single threshold segmentation method that fails to achieve high-precision extraction of surface water in complex terrain areas, this study introduces a multiple threshold water detection rule (MTWDR) method to improve water extraction results. This method uses the multi-band reflectance characteristics of ground features to construct a water index and combines brightness features with the Otsu algorithm to eliminate interference from highly reflective ground features like ice, snow, bright material buildings, and clouds. The Yunan–Guizhou Plateau was selected as the study area due to its complex terrain and multiple types of surface water, and experiments were conducted using Sentinel-2 data on the Google Earth Engine (GEE). The results demonstrate that: (1) The proposed method achieves an overall accuracy of 94.08% and a kappa coefficient of 0.8831 in mountainous areas. In urban areas, the overall accuracy reaches 95.15%, accompanied by a kappa coefficient of 0.8945. (2) Compared to five widely used water indexes and rules, the MTWDR method improves accuracy by more than 3%. (3) It effectively overcomes interference from highly reflective ground features while maintaining the integrity and accuracy of water boundary extraction. In conclusion, the proposed method enhances extraction accuracy across different types of surface water within complex terrain areas, and can provide significant theoretical implications and practical value for researching and applying surface water resources.

Rethinking Logic Minimization for Tabular Machine Learning

Tabular datasets can be viewed as logic functions that can be simplified using two-level logic minimization to produce minimal logic formulas in disjunctive normal form (DNF), which in turn can be readily viewed as an explainable decision rule set for binary classification. However, there are two problems with using logic minimization for tabular machine learning. First, tabular datasets often contain overlapping examples that have different class labels, which have to be resolved before logic minimization can be applied since logic minimization assumes <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">consistent</i> logic functions. Second, even without inconsistencies, logic minimization alone generally produces complex models with poor generalization because it exactly fits all data points, which leads to detrimental <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">overfitting</i> . How best to remove training instances to eliminate inconsistencies and overfitting is highly non-trivial. In this paper, we propose a novel statistical framework for removing these training samples so that logic minimization can become an effective approach to tabular machine learning. Using the proposed approach, we are able to obtain comparable performance as gradient boosted and ensemble decision trees, which have been the winning hypothesis classes in tabular learning competitions, but with human-understandable explanations in the form of decision rules. To our knowledge, neither logic minimization nor explainable decision rule methods have been able to achieve state-of-the-art performance before in tabular learning problems.

The ReadFree tool for the identification of poor readers: a validation study based on a machine learning approach in monolingual and minority-language children

In this study, we validated the “ReadFree tool”, a computerised battery of 12 visual and auditory tasks developed to identify poor readers also in minority-language children (MLC). We tested the task-specific discriminant power on 142 Italian-monolingual participants (8–13 years old) divided into monolingual poor readers (N = 37) and good readers (N = 105) according to standardised Italian reading tests. The performances at the discriminant tasks of the “ReadFree tool” were entered into a classification and regression tree (CART) model to identify monolingual poor and good readers. The set of classification rules extracted from the CART model were applied to the MLC’s performance and the ensuing classification was compared to the one based on standardised Italian reading tests. According to the CART model, auditory go-no/go (regular), RAN and Entrainment100bpm were the most discriminant tasks. When compared with the clinical classification, the CART model accuracy was 86% for the monolinguals and 76% for the MLC. Executive functions and timing skills turned out to have a relevant role in reading. Results of the CART model on MLC support the idea that ad hoc standardised tasks that go beyond reading are needed.

An Advanced Fitness Function Optimization Algorithm for Anomaly Intrusion Detection Using Feature Selection

Cyber-security systems collect information from multiple security sensors to detect network intrusions and their models. As attacks become more complex and security systems diversify, the data used by intrusion-detection systems becomes more dimensional and large-scale. Intrusion detection based on intelligent anomaly detection detects attacks based on machine-learning classification models, soft computing, and rule sets. Feature-selection methods are used for efficient intrusion detection and solving high-dimensional problems. Optimized feature selection can maximize the detection model performance; thus, a fitness function design is required. We proposed an optimization algorithm-based feature-selection algorithm to improve anomaly-detection performance. We used a genetic algorithm and proposed an advanced fitness function that finds the most relevant feature set, increasing the detection rate, reducing the error rate, and enhancing analysis speed. An improved fitness function for the selection of optimized features is proposed; this function can address overfitting by solving the problem of anomaly-detection performance from imbalanced security datasets. The proposed algorithm outperformed other feature-selection algorithms. It outperformed the PCA and wrapper-DR methods, with 0.99564 at 10%, 0.996455 at 15%, and 0.996679 at 20%. It performed higher than wrapper-DR by 0.95% and PCA by 3.76%, showing higher differences in performance than in detection rates.

A New Measure for Determining the Equivalent Symmetry of Decomposed Subsystems from Large Complex Cyber–Physical Systems

In this paper, we propose a new consistency measurement for classification rule sets that is based on the similarity of their classification abilities. The similarity of the classification abilities of the two rule sets is evaluated though the similarity of the corresponding partitions of the feature space using the different rule sets. The proposed consistency measure can be used to measure the equivalent symmetry of subsystems decomposed from a large, complex cyber–physical system (CPS). It can be used to verify whether the same knowledge is obtained by the sensing data in the different subsystems. In the experiments, five decision tree algorithms and eighteen datasets from the UCI machine learning repository are employed to extract the classification rules, and the consistency between the corresponding rule sets is investigated. The classification rule sets extracted from the use of the C4.5 algorithm on the electrical grid stability dataset have a consistency of 0.88, which implies that the different subsystems contain almost equivalent knowledge about the network stability.

Algorithmic TCAMs: Implementing Packet Classification Algorithms in Hardware

The adoption of cloud computing and the deployment of new services require more and more computing and networking resources in large data centers. In particular, there is a need for faster, rich featured switches. Today, switching integrated circuits can handle more than 50 Tb/s and are expected to reach 100 Tb/s next year. This trend is likely to continue, approaching petabit capacities in a few years. At the same time, switches need to support more advanced packet classification features. This poses many challenges to the designers of switching integrated circuits, creating a need for innovations to move forward. One of those challenges is to support programmable packet classification with large rulesets at wire speed using limited silicon footprint. Unfortunately, traditional solutions like ternary content addressable memories (TCAMs) have high cost in area and power, and therefore are not a viable option for large on-chip rulesets, so new alternatives are needed. In the last two decades, researchers have extensively studied the problem of packet classification, proposing many algorithms based on standard memories, mostly targeting software implementations. These packet classification algorithms could be a solution to enable large on-chip classification rulesets using standard memories. However, porting the standard packet classification algorithms to a hardware implementation is not a trivial task, due to the different requirements and the different kinds of memory and computing resources available on application-specific integrated circuit (ASIC) switching chips. This article describes a packet classification design tailored for high-speed switching integrated circuits, which is currently used in the NVIDIA® Mellanox® Spectrum® switching ASIC product line, illustrating how to solve the mismatch between the usual software-based classification algorithms and the specific requirements of a hardware implementation.

Sparse concordance‐based ordinal classification

AbstractOrdinal classification is an important area in statistical machine learning, where labels exhibit a natural order. One of the major goals in ordinal classification is to correctly predict the relative order of instances. We develop a novel concordance‐based approach to ordinal classification, where a concordance function is introduced and a penalized smoothed method for optimization is designed. Variable selection using the penalty is incorporated for sparsity considerations. Within the set of classification rules that maximize the concordance function, we find optimal thresholds to predict labels by minimizing a loss function. After building the classifier, we derive nonparametric estimation of class conditional probabilities. The asymptotic properties of the estimators as well as the variable selection consistency are established. Extensive simulations and real data applications show the robustness and advantage of the proposed method in terms of classification accuracy, compared with other existing methods.

RuleCOSI+: Rule extraction for interpreting classification tree ensembles

Despite the advent of novel neural network architectures, tree-based ensemble algorithms such as random forests and gradient boosting machines still prevail in many practical machine learning problems in manufacturing, financial, and medical domains. However, tree ensembles have the limitation that the internal decision mechanisms of complex models are difficult to understand. Therefore, we present a post-hoc interpretation approach for classification tree ensembles. The proposed method, RuleCOSI+, extracts simple rules from tree ensembles by greedily combining and simplifying their base trees. Compared with its previous version, RuleCOSI, this new version can be applied to both bagging (e.g., random forest, RF) and boosting ensembles (e.g., gradient boosting machines, GBM) and run much faster for ensembles with hundreds of trees. To assess the performance and applicability of the method, empirical experiments were conducted using two bagging algorithms and four gradient boosting algorithms over 33 datasets. RuleCOSI+ could generate the best classification rulesets in terms of F-measure together with RuleFit for RF and GBM models of the datasets among five ensemble simplification algorithms, but the rulesets of RuleCOSI+ had, on average, less than half the size of those of RuleFit. Moreover, RuleCOSI+ had the best antecedent uniqueness rate (“uniq”) among the five algorithms, and had also ranked high in the number of rules (“Nrules”) and the rule reduction rate (“redu”). In addition, the proposed method could reduce generalization errors in the simplified rulesets to obtain, on average, slightly better classification errors than original models of two bagging and three gradient boosting algorithms except CATBoost.

Data-Based Method for the Implementation Planning of Engineering Changes in the Automotive Industry

AbstractEach year, automotive OEMs implement a variety of Engineering Changes (ECs) in their production. In the timing of ECs, different KPIs are often in conflict with one another or even unknown to the OEMs. Therefore, OEMs struggle to identify the optimal date to implement an EC. This paper presents a method to determine the cost-optimal implementation date for each EC, considering time, cost, and quality KPIs based on a new EC classification rule-set. To evaluate the presented method, case-studies at a German automotive OEM were performed, two of which are discussed.

Analyzing the Relationship between the Personality Traits and Drug Consumption (Month-based user Definition) using Rough Sets Theory

There is no doubt that the use of drugs has significant consequences for society, it introduces risk into the human life and causing earlier mortality and morbidity. Being a conscientious member of society, we must go ahead to prevent these young minds from life-threatening addiction. Owing to the computational complexity of wrapper approaches, the poor performance of filtering techniques, and the classifier dependency of embedded approaches, artificial intelligence and machine learning systems can provide useful tools for raising the prediction rate of drug users. Recently, the psychologists approved the recent personality traits Five Factor Model (FFM) for understanding human individual differences. The aim of this work is to propose a rough sets theory based method to investigate the relationship between drug user/non-user (month-based user definition) and the personality traits. The data of five factor personality profiles, impulsivity, sensation-seeking and biographical information of users of 21 different types of legal and illegal drugs are used to fetch all reducts and finally a set of classification rules are created to predict the drug user/non-user(month-based user definition). The outcomes demonstrate the novelty of the current work which can be summarized as The set of generalized classification rules which pronounced with logic functions build a knowledge base with excellent accuracy to analyze drug misuse successfully and may be worthy in many applications.

Cost Control of Treatment for Cerebrovascular Patients Using a Machine Learning Model in Western China.

Background Cerebrovascular disease has been the leading cause of death in China since 2017, and the control of medical expenses for these diseases is an urgent issue. Diagnosis-related groups (DRG) are increasingly being used to decrease the costs of healthcare worldwide. However, the classification variables and rules used vary from region to region. Of these variables, the question of whether the length of stay (LOS) should be used as a grouping variable is controversial. Aim To identify the factors influencing inpatient medical expenditure in cerebrovascular disease patients. The performance of two sets of classification rules, and the effects of the extent of control of unreasonable medical treatment, were compared, to investigate whether the classification variables should include LOS. Methods Data from 45,575 inpatients from a Healthcare Security Administration of a city in western China were used. Kruskal–Wallis H tests were used for single-factor analysis, and multiple linear stepwise regression was used to determine the main factors. A chi-squared automatic interaction detector (CHAID) algorithm was built as a decision tree model for grouping related data. The intensity of oversupply of service was controlled step by step from 10% to 100%, and the performance was calculated for each group. Results The average hospitalization cost was 1,284 US dollars, and the total was 51.17 million US dollars. Of this, 43.42 million were paid by the government, and 7.75 million were paid by individuals. Factors including gender, age, type of insurance, level of hospital, LOS, surgery, therapeutic outcomes, main concomitant disease, and hypertension significantly influenced inpatient expenditure (P < 0.05). Incorporating LOS, the patients were divided into seven DRG groups, while without LOS, the patients were divided into eight DRG groups. More clinical variables were needed to achieve good results without LOS. Of the two rule sets, smaller coefficient of variation (CV) and a lower upper limit for patient costs were found in the group including LOS. Using this type of economic control, 3.35 million US dollars could be saved in one year.

Patient-Based Prediction Algorithm of Relapse after Allo-HSCT for Acute Leukemia Using Machine Learning Approach

Background:Although allogeneic hematopoietic stem-cell transplantation (allo-HSCT) is a curative therapy for patients with high risk acute leukemia (AL), a method of reduction in relapse after allo-HSCT remains to be elucidated. Prognostic factors for allo-HSCT for AL, such as the Disease Risk Index (DRI), donor and conditioning type, hematopoietic cell transplantation -specific comorbidity index (HCT-CI), have been determined with conventional statistic methods like uni- and multivariate analysis. However, patient-based prediction of relapse, which would be useful for bedside decision making or for a protocol of diversified allo-HSCT, is difficult because all of these factors are related with each other in the patient.Alternating decision tree (ADTree) is a successful classification method that combines decision trees with the predictive accuracy of boosting into a set of interpretable classification rules. It is one part of the machine learning (ML) approach based on artificial intelligence and can handle multiple factors simultaneously. Recently, some ML approaches like ADTree have been highlighted as promising approaches for the development of new prediction models.Purpose:To construct a patient-based prediction algorithm of AL relapse within 1-year after allo-HSCT to help in deciding on therapy options to prevent relapse.Patients and Method:Our analysis was a retrospective, supervised data mining learning study that included 223 AL patients (acute myelogenous leukemia; AML n=132, acute lymphoblastic leukemia; ALL n=91). They received allo-HSCT at Niigata University Hospital (n=144) and Nagaoka Red Cross Hospital (n=79) from 1990 to 2016. The median age at HSCT was 38 years old (range 18-71). The median follow-up for living patients was 61 months (range 12-223). Myeloablative conditioning was used in 174 (78%) and reduced-intensity in 49 (22%) patients. Donors were related for 97 (43.5%) and unrelated for 126 (56.5%) patients. Grafts were peripheral blood stem cells in 49 (22.0%), bone marrow in 128 (57.4%) and cord blood in 46 (20.6%) patients. According to the DRI for low, intermediate, high and very high-risk, there were 12 (5.4%), 124 (55.6%), 56 (25.1%) and 31 (13.9%) patients, respectively.ADTree was performed using the WEKA software (Ver.3.9.1, Machine Learning Group at the University of Waikato, New Zealand). The algorithm model was trained and tested using 10-fold cross validation on the training data set (Niigata group) and validated again on the validation data set (Nagaoka group). The model was evaluated for prediction accuracy and the area under curve (AUC) of the receiver operating characteristics (ROC) analysis, which discriminates the true prediction rate from the false prediction rate, was determined.Result:The 1-year relapse rate and 1-year overall survival of all patients were 35.5% and 69.7%, respectively. The ADTree model selected 10 clinical variables based on the statistical values during analysis. As the accuracy rate and the AUC of prediction of this algorithm were 76.2% and 0.690 in the training data set and 77.2% and 0.714 in the validation data set, respectively (Table 1), this model has high accuracy and generalization ability for predicting AL relapse.Figure 1 shows the graphical output of the ADTree prediction model. Each score indicates the prediction node weight (NW); NW<0 lower risk and NW>0 higher risk to relapse. In the same disease stage categories (DRI; very high), AML patients had lower risk (NW, -0.133) than ALL patients (NW, 0.568). The final judgment of AL relapse prediction within 1-year was as follows; total NW sum> 0 was predicted relapse and <0 was no predicted relapse, respectively.For example, in the case of a patient (DRI; high, under 40 years old and HCT-CI ≦2) who received CBT (cord blood transplantation) with total body irradiation (TBI), the NW sum was -1.165<0, which indicates no predicted relapse. On the other hand, if the same patient received allo-HSCT not with CBT and without TBI, his NW sum would be 0.116>0, which indicates predicted relapse. (Table 2)Conclusion:This algorithm has high prediction accuracy and generalization ability. Our analysis suggested that the AL relapse within 1-year may be changed by therapy options. The ML approach like ADTree is promising to construct a new patient-based prognostic prediction algorithm and it may be useful in bed-side decision-making for diversified allo-HSCT protocols. [Display omitted] DisclosuresNo relevant conflicts of interest to declare.

The Accuracy of Trade Classification Systems on the Foreign Exchange Market: Evidence from the RUB/USD Market

To the best of our knowledge we are the first to test a broad set of trade classification rules on the foreign exchange interbank market. A unique data set on the Russian Rouble/US Dollar trade includes the true trade initiator. The modified EMO (Ellis, Michaely and O'Hara) rule is currently the best choice at classifying trades. When quote data is not present, the tick rule yields a considerably lower accuracy. Yearly variations in the accuracy can be attributed to the difference in the location where trades occurred. Not surprisingly, trades executed at the quotes are the most informative.

Multilevel Extraction of Vegetation Type Based on Airborne LiDAR Data

Precise determination of vegetation type is important in remote sensing of the ecological environment. Many studies have explored ecosystem structure on explicit spatial scales using specific remote sensing data, but few studies have considered vegetation information extraction at various landscape levels using LiDAR-derived raster layers. This study determined vegetation information based on LiDAR-derived canopy height model (CHM) and LiDAR-derived forest canopy coverage (FCC) from the typical natural secondary forest of Maoershan Experimental Forest Farm (Northeast China). Geographic object-based image analysis was adopted for all experiments. Optimal classification characteristics and thresholds were determined and classification rule sets established for vegetation type extraction at four levels: vegetation-not-vegetation, vegetation type, forest type, and canopy—canopy gap. We compared and analyzed the capability of LiDAR-derived raster layers extraction to describe vegetation features at these four levels. An area-based assessment method was used for accuracy verification. The research showed that physical information such as vegetation height and canopy density provided by LiDAR point cloud is effective for extracting vegetation characteristics and categories. Moreover, results showed that high-resolution LiDAR-derived raster layers could provide more detailed vegetation information. This study represents a reference for data selection and mapping strategies for hierarchical and multiscale vegetation type extraction.

Preventing the generation of inconsistent sets of crisp classification rules

In recent years, the interest in interpretable classification models has grown. One of the proposed ways to improve the interpretability of a classification model based on collections of crisp rules is to use sets (unordered collections) instead of lists (ordered collections). One of the problems associated with sets is that multiple rules may cover a single instance but predict different classes for it, thus requiring a conflict resolution strategy. In this work, we propose two algorithms capable of finding feature-space regions inside which any created rule would be consistent with the already existing rules, preventing inconsistencies from arising. Our algorithms, named CFSGS and CFSBE, do not generate classification rules nor classification models but are instead meant to enhance algorithms that do so, such as Learning Classifier Systems. We analyzed both algorithms from a theoretical perspective and conducted experiments with a proof of concept evolutionary algorithm that employs CFSBE. The experiments suggest that using CFSBE as an embedded tool does incur a computational overhead, but such cost is not prohibitive.

Decision Tree Algorithms for Developing Rulesets for Object-Based Land Cover Classification

Decision tree (DT) algorithms are important non-parametric tools used for land cover classification. While different DTs have been applied to Landsat land cover classification, their individual classification accuracies and performance have not been compared, especially on their effectiveness to produce accurate thresholds for developing rulesets for object-based land cover classification. Here, the focus was on comparing the performance of five DT algorithms: Tree, C5.0, Rpart, Ipred, and Party. These DT algorithms were used to classify ten land cover classes using Landsat 8 images on the Copperbelt Province of Zambia. Classification was done using object-based image analysis (OBIA) through the development of rulesets with thresholds defined by the DTs. The performance of the DT algorithms was assessed based on: (1) DT accuracy through cross-validation; (2) land cover classification accuracy of thematic maps; and (3) other structure properties such as the sizes of the tree diagrams and variable selection abilities. The results indicate that only the rulesets developed from DT algorithms with simple structures and a minimum number of variables produced high land cover classification accuracies (overall accuracy &gt; 88%). Thus, algorithms such as Tree and Rpart produced higher classification results as compared to C5.0 and Party DT algorithms, which involve many variables in classification. This high accuracy has been attributed to the ability to minimize overfitting and the capacity to handle noise in the data during training by the Tree and Rpart DTs. The study produced new insights on the formal selection of DT algorithms for OBIA ruleset development. Therefore, the Tree and Rpart algorithms could be used for developing rulesets because they produce high land cover classification accuracies and have simple structures. As an avenue of future studies, the performance of DT algorithms can be compared with contemporary machine-learning classifiers (e.g., Random Forest and Support Vector Machine).

Detection of the helper types from story in multimedia

A story unfolds through the relationships formed by the major and minor characters. The major characters are central to the main plot of the story, creating and resolving the conflicts. In contrast, the minor characters tend to play the role of awakening the major characters or solving their problems. Although the minor characters appear less frequently than the major characters, they play crucial roles in that they create tension, provide clues that lead to a solution, and so on, thereby making the story more interesting and engaging. Therefore, an analysis of the minor characters is essential for analyzing the entire story. However, the existing character analysis methods such as Character-net and RoleNet are not entirely suitable for analyzing the roles of minor characters in a story, even though they are adequate for classifying whether a character is one of the major characters or a minor character based on an analysis of the cumulative outcomes of the story. For an accurate analysis of the roles of minor characters, there is a need to study the pattern in which the characters appear as the story progresses, not to analyze the story based on cumulative outcomes. Accordingly, this paper proposes a method of classifying minor characters as either a mentor or a best friend, based on a three-dimensional visualization of Character-net which can depict the story progress. In order to make such distinctions, a set of classification rules based on appearance patterns and density was proposed, and forty characters chosen from twenty-five movies were examined for the purpose of performance evaluation. The results showed that the proposed method had a significant classification performance with an F1-measure of 0.565 with respect to the forty characters.

Development of information and analytical tools based on adaptive classifier Cascade ARTMAP

The article deals with methods of teaching incremental classifiers capable to adapt to incoming online streaming data, using the example of environmental monitoring systems. It is proposed to use Cascade ARTMAP neural network architecture, compatible with symbolic representation based on IF- THEN rules. The knowledge gained during Cascade ARTMAP network training can be transformed into a compact set of decisive rules for classification of source data that can be easily interpreted by experts in the field of environmental assessment.

Reliability-as-a-Service for bearing risk assessment investigated with advanced mathematical models

As a key player in bearing service life, the lubricant chemistry has a profound effect on bearing reliability. To increase the reliability of bearings, an Industrial Analytics solution is proposed for proactive condition monitoring and this is delivered via a Reliability-as-a-Serviceapplication. The performance predictions of bearings rely on customized algorithms with the main focus on digitalizing lubricant chemistry; the principles behind these processes are outlined in this study. Subsequently, independent testing is performed to confirm the ability of the presented Industrial Analytics solution for such predictions. By deciphering the chemical compounds of lubricants and characteristics of the interface, the Industrial Analytics solution delivers a precise bearing reliability assessment a priori to predict service life of the operation. Bearing tests have shown that the classification system of this Industrial Analytics solution is able to predict 12 out of 13 bearing failures (92%). The described approach provides a proactive bearing risk classification that allows the operator to take immediate action in reducing the failure potential during smooth operation - preventing any potential damage from occurring. For this purpose, a mathematical model is introduced that derives a set of classification rules for oil lubricants, based on linear binary classifiers (support vector machines) that are applied to the chemical compound’s mixture data.

Anomaly explanation with random forests

Anomaly detection has become an important topic in many domains with many different solutions proposed until now. Despite that, there are only a few anomaly detection methods trying to explain how the sample differs from the rest. This work contributes to filling this gap because knowing why a sample is considered anomalous is critical in many application domains. The proposed solution uses a specific type of random forests to extract rules explaining the difference, which are then filtered and presented to the user as a set of classification rules sharing the same consequent, or as the equivalent rule with an antecedent in a disjunctive normal form. The quality of that solution is documented by comparison with the state of the art algorithms on 34 real-world datasets.