Rule Extraction Research Articles

Ensemble Pretrained Convolutional Neural Networks for the Classification of Insulator Surface Conditions

Overhead transmission line insulators are non-conductive materials that separate conductors from grounded transmission towers. Once in operation, they frequently experience environmental pollution and electrical or mechanical stress. Since adverse operational conditions can lead to insulation failure, regular inspections are essential to prevent power outages. To this end, this paper proposes a novel technique based on deep convolutional neural networks (CNNs) to classify high-voltage insulator surface conditions based on their image. Successful applications of CNNs in computer vision have led to several pretrained architectures for image classification. To use these pretrained models, a practitioner typically fine-tunes and selects one final model via a model selection stage and discards all other models. In contrast with many existing studies that use such a “winner-takes-all” approach, here, we identify the best subset of seven popular pretrained CNN architectures that are combined by soft voting to form an ensemble classifier. From a machine learning (ML) perspective, this focus is warranted because the convolutional base of each pretrained architecture operates as a feature extractor and an ensemble of them works as a combination of various feature extraction rules. Our numerical experiments demonstrate the advantage of the identified ensemble model to individual pretrained architectures.

Learning rule in MFR pulse sequence for behavior mode prediction

Radar behavior prediction is an important task in the field of electronic reconnaissance. For the extensive applied multi-function radar (MFR), which can flexibly transition between various work modes and make certain statistical rule of these radar behaviors exist in the signal sequence. Most of existing radar emission prediction methods are inapplicable to the non-cooperative scenario, since the labeled sequence samples are hard to obtain. To solve this challenge, an unsupervised framework is proposed for learning the behavior rule from the pulse sequence and predicting the radar mode in this paper. The framework includes three modules of sequence segmentation for mode switch boundaries detection, segment clustering for behavior mode recognition, and mode prediction for behavior rule extraction. The application of this framework can predict state and numerical values of next mode at the same time. Experimental results demonstrate that the proposed framework has a considerable prediction performance and shows good robustness under the non-ideal conditions.

Hybrid lane change strategy of autonomous vehicles based on SOAR cognitive architecture and deep reinforcement learning

Research on lane change strategies for autonomous vehicles holds paramount importance in optimizing traffic flow efficiency, enhancing driving safety, and adapting to complex traffic environments. While numerous rule-based or machine-learning approaches have been explored to tackle the challenge of lane change on highways, they frequently exhibit limited performance owing to the complexity of driving environments. This study proposes a novel lane change strategy for autonomous vehicles, which utilizes a hybrid framework integrating the SOAR cognitive architecture and deep reinforcement learning (DRL) to address the lane change challenge on highways. First, we introduce a rule extraction algorithm, the RuleCOSI+, which is based on tree ensemble algorithms, designed to extract concise lane change rules from large-scale human driving data. These straightforward rules, together with traffic regulations and safety rules, constitute the long-term memory of the SOAR cognitive architecture, enabling transparent decision-making processes. Next, by analyzing the clipping mechanism of the proximal policy optimization (PPO) algorithm, we propose an Adaptive Clipping PPO (ACPPO) algorithm which is based on the importance of samples. This algorithm adopts different clipping strategies for SOAR samples and ACPPO samples during the training process, enabling the algorithm to more effectively utilize samples with different levels of importance. Then, we propose a hybrid decision-making algorithm: SOAR-ACPPO, which combines the SOAR cognitive architecture with the ACPPO algorithm. This algorithm leverages SOAR’s prior knowledge to effectively and safely guide agent learning. Finally, by selecting appropriate intervention probability and weaning strategy, the system avoids inappropriate knowledge intervention and ensures adequate environment exploration. Simulation experiments conducted using the CARLA simulator illustrate that the proposed strategy not only improves model learning efficiency but also enhances driving efficiency and safety. Additionally, it demonstrates a certain degree of human-like characteristics and interpretability.

Analyzing the Interplay Between Competition State Anxiety, Motor Motivation, and Coping Styles Among Adolescent Track and Field Athletes

The relationship between competition state anxiety, motor motivation and coping styles of adolescent track and field athletes in China was investigated using interview and questionnaire research methods. The results showed that the mean scores of cognitive state anxiety and somatic state anxiety were lower in junior track and field athletes who had entered the echelon for a short period of time than in older athletes, and the opposite was true for state self-confidence; there were highly significant differences and significant differences in the identity regulation and introjection regulation dimensions of motor motivation; and there were significant differences in the focused problem-solving coping dimension of coping style. This paper proposes an algorithm for classifying athletic visual mirrors based on sequential model mining. This paper focuses on two issues – feature extraction and definition of semantic rules. In the feature extraction stage, the track and field video footage is automatically segmented into a series of identifiable sequences of athletic events, and then each type of behavioral event is identified using a mechanically learned algorithm. There were no significant differences between the three age groups in terms of race state anxiety, identity regulation and introjection regulation, and no significant differences in coping styles. There were no significant differences in the anxiety of competition status, motivation and coping styles among youth athletes of different sport levels. The results showed the effectiveness of the present algorithm for classifying track and field video cameras.

A novel reservoir dispatching rules extraction framework based on hybrid embedding informer

Accurately capturing reservoir dispatching rules is crucial for mitigating carbon emissions and preventing flooding and inundation. However, the short operational duration of certain reservoirs poses challenges in acquiring sufficient data samples for reservoir rule simulation. Furthermore, the nonlinear and fluctuating characteristics of reservoir dispatching rules present obstacles to their precise prediction. To address these challenges, this study proposes a hybrid embedding-based Informer (Heformer) dispatching rules extraction framework. Initially, extensive data series are employed for modeling to acquire a comprehensive sample dataset. Subsequently, the Heformer model, based on the Informer architecture and integrating the Dimension-Segment-Wise (DSW) strategy and feature representation strategy, is introduced for capturing reservoir dispatching rules. To assess performance of the proposed model, it is compared against several recognized state-of-the-art models. Experimental results demonstrate that the proposed method effectively captures the operational characteristics of reservoirs, mitigates bias impact in reservoir prediction and operation, and provides a reference for extracting dispatching rules for reservoirs.

Multi-Scale Variation in Surface Water Area in the Yellow River Basin (1991–2023) Based on Suspended Particulate Matter Concentration and Water Indexes

Surface water is a crucial part of terrestrial ecosystems and is crucial to maintaining ecosystem health, ensuring social stability, and promoting high-quality regional economic development. The surface water in the Yellow River Basin (YRB) has a high sediment content and spatially heterogeneous sediment distribution, presenting a significant challenge for surface water extraction. In this study, we first analyze the applicability of nine water indexes in the YRB by using the Landsat series images (Landsat 5, 7, 8) and then examine the correlation between the accuracy of the water indexes and suspended particulate matter (SPM) concentrations. On this basis, we propose a surface water extraction method considering the SPM concentrations (SWE-CSPM). Finally, we examine the dynamic variations in the surface water in the YRB at four scales: the global scale, the secondary water resource zoning scale, the provincial scale, and the typical water scale. The results indicate that (1) among the nine water indexes, the MBWI has the highest water extraction accuracy, followed by the AWEInsh and WI2021, while the NDWI has the lowest. (2) Compared with the nine water indexes and the multi-index water extraction rule method (MIWER), the SWE-CSPM can effectively reduce the commission errors of surface water extraction, and the water extraction accuracy is the highest (overall accuracy 95.44%, kappa coefficient 90.62%). (3) At the global scale, the maximum water area of the YRB shows a decreasing trend, but the change amount is small. The permanent water area shows an uptrend, whereas the seasonal water area shows a downtrend year by year. The reason may be that the increase in surface runoff and the construction of reservoir projects have led to the transformation of some seasonal water into permanent water. (4) At the secondary water resource zoning scale, the permanent water area of other secondary water resource zonings shows an increasing trend in different degrees, except for the Interior Drainage Area. (5) At the provincial scale, the permanent water area of all provinces shows an uptrend, while the seasonal water areas show a fluctuating downtrend. The maximum water area of Shandong, Inner Mongolia Autonomous Region, and Qinghai increases slowly, while the other provinces show a decreasing trend. (6) At the typical water scale, there are significant differences in the water area variation process in Zhaling Lake, Eling Lake, Wuliangsuhai, Hongjiannao, and Dongping Lake, but the permanent water area and maximum water area of these waters have increased over the past decade. This study offers significant technical support for the dynamic monitoring of surface water and helps to deeply understand the spatiotemporal variations in surface water in the YRB.

CapsRule: Explainable Deep Learning for Classifying Network Attacks.

Despite the potential deep learning (DL) algorithms have shown, their lack of transparency hinders their widespread application. Extracting if-then rules from deep neural networks is a powerful explanation method to capture nonlinear local behaviors. However, existing rule extraction methods suffer from inefficiency, incomprehensibility, infidelity, and not scaling well. Concerning security applications, they are not optimized regarding the decision boundary, data types and ranges, classification tasks, and dataset size. In this article, we propose CapsRule, an effective and efficient rule-based DL explanation method dedicated to classifying network attacks. It extracts high-fidelity rules from the feed-forward capsule network that explains how an input sample is classified. Using precomputed coupling coefficients, the training phase overlaps the rule extraction process to increase efficiency. The activation vector of a capsule can represent semantic intelligence about the attributes of the input sample. The rules extracted from CapsRule address the major concerns of network attack detection. The rules: 1) approximate the nonlinear decision boundary of the underlying data; 2) reduce the number of false positives significantly; 3) increase transparency; and 4) help find errors and noise in the data. We evaluate CapsRule on the CICDDoS2019 dataset that contains over a million of the most advanced Distributed Denial-of-Service (DDoS) attacks. The extensive evaluation shows that it generates accurate, high-fidelity, and comprehensible rules. CapsRule achieves an average accuracy of 99.0% and a false positive rate of 0.70% for reflection- and exploitation-based attacks. We verify that the learned features from the rulesets match our domain-specific knowledge. They also help find flaws in the dataset generation process and erroneous patterns caused by attack simulators.

Fuzzy Adaptive Knowledge-Based Inference Neural Networks: Design and Analysis.

A novel fuzzy adaptive knowledge-based inference neural network (FAKINN) is proposed in this study. Conventional fuzzy cluster-based neural networks (FCBNNs) suffer from the challenge of a direct extraction of fuzzy rules that can capture and represent the interclass heterogeneity and intraclass homogeneity when the data possess complex structures. Moreover, the capability of the cluster-based rule generator in FCBNNs may decrease with the increase of data dimensionality. These drawbacks impede the generation of desired fuzzy rules, and affect the inference results depending on the fuzzy rules, thereby limiting their generalization ability. To address these drawbacks, an adaptive knowledge generator (AKG), consisting of the observation paradigm (OP) and clustering strategy (CS), is effectively designed to improve the generalization ability in FAKINN. The OP distills the characteristic information (CI) from data to highlight the homogeneity and heterogeneity of objects, and the CS, viz., the weighted condition-driven fuzzy clustering method (WCFCM), is proposed to summarize the CI to construct fuzzy rules. Moreover, the feedback between the OP and CS can control the dimensionality of CI, which endows FAKINN with the potential to tackle high-dimensional data. The main originality of the study focuses on the AKG and WCFCM that are proposed to develop the structural design methodology of FNNs. The performance of FAKINN is evaluated on various benchmarks with 27 comparative methods, and two real-world problems are adopted to validate its effectiveness. Experimental results show that FAKINN outperforms the comparison methods.

Visual exploration of multi-dimensional data via rule-based sample embedding

We propose an approach to learning sample embedding for analyzing multi-dimensional datasets. The basic idea is to extract rules from the given dataset and learn the embedding for each sample based on the rules it satisfies. The approach can filter out pattern-irrelevant attributes, leading to significant visual structures of samples satisfying the same rules in the projection. In addition, analysts can understand a visual structure based on the rules that the involved samples satisfy, which improves the projection’s pattern interpretability. Our research involves two methods for achieving and applying the approach. First, we give a method to learn rule-based embedding for each sample. Second, we integrate the method into a system to achieve an analytical workflow. Cases on real-world dataset and quantitative experiment results show the usability and effectiveness of our approach.

Automated machine learning with interpretation: A systematic review of methodologies and applications in healthcare

AbstractMachine learning (ML) has achieved substantial success in performing healthcare tasks in which the configuration of every part of the ML pipeline relies heavily on technical knowledge. To help professionals with borderline expertise to better use ML techniques, Automated ML (AutoML) has emerged as a prospective solution. However, most models generated by AutoML are black boxes that are challenging to comprehend and deploy in healthcare settings. We conducted a systematic review to examine AutoML with interpretation systems for healthcare. We searched four databases (MEDLINE, EMBASE, Web of Science, and Scopus) complemented with seven prestigious ML conferences (AAAI, ACL, ICLR, ICML, IJCAI, KDD, and NeurIPS) that reported AutoML with interpretation for healthcare before September 1, 2023. We included 118 articles related to AutoML with interpretation in healthcare. First, we illustrated AutoML techniques used in the included publications, including automated data preparation, automated feature engineering, and automated model development, accompanied by a real‐world case study to demonstrate the advantages of AutoML over classic ML. Then, we summarized interpretation methods: feature interaction and importance, data dimensionality reduction, intrinsically interpretable models, and knowledge distillation and rule extraction. Finally, we detailed how AutoML with interpretation has been used for six major data types: image, free text, tabular data, signal, genomic sequences, and multi‐modality. To some extent, AutoML with interpretation provides effortless development and improves users' trust in ML in healthcare settings. In future studies, researchers should explore automated data preparation, seamless integration of automation and interpretation, compatibility with multi‐modality, and utilization of foundation models.

A hybrid approach to extract conceptual diagram from software requirements

Employing rules for the automatic extraction of conceptual diagrams from software requirements has been in practice for some time. However, considering only rules for extraction makes the system complex to handle. Moreover, the rules are predominantly based on the syntactic structure such as Part of Speech tags along with Dependency Grammar of sentences and rarely on semantics. In this paper, we propose to use a probabilistic approach in configuration with the rule-based technique and the Word embeddings to preserve the semantics of the sentence. Hence, reduces the complexity of the extraction procedure. Further, we advocate the use of a divide-and-conquer policy of extraction instead of extracting classes for one entire use case description. We extract the class diagram from small use cases and then merge it to obtain the class diagram. As generated class diagram corresponding to small use cases can be utilized in another similar software design, thus, it increases the scalability and decreases the extraction time. The proposed hybrid approach integrates the knowledge from the experiences. Thus, the proposed approach achieved 90% as F1-score whereas the F1-Score for the existing methods ranged between 79-88%. The proposed hybrid approach also shows a 19.44% reduction in terms of the number of iterations performed to carry out extraction procedures for individual use cases. Hence, reduces the extraction procedure complexity.

Emergency Department Length of Stay Classification Based on Ensemble Methods and Rule Extraction.

This study employs machine learning techniques to identify factors that influence extended Emergency Department (ED) length of stay (LOS) and derives transparent decision rules to complement the results. Leveraging a comprehensive dataset, Gradient Boosting exhibited marginally superior predictive performance compared to Random Forest for LOS classification. Notably, variables like triage acuity and the Elixhauser Comorbidity Index (ECI) emerged as robust predictors. The extracted rules optimize LOS stratification and resource allocation, demonstrating the critical role of data-driven methodologies in improving ED workflow efficiency and patient care delivery.

Visual analysis of oceanic data for marine ecosystems

We present a data visualization approach that allows users to interactively analyse sequential movement patterns and automatically extract rules for marine animal behaviour. This enhances the ability of oceanic experts to understand marine ecosystems. Traditional time-series analysis, basic statistical methods, and single-view visualizations often fail to capture the intricate spatial and temporal dynamics of marine ecosystems. Our visual analytics approach, that leverages sequential pattern mining and rule mining techniques, integrates SPMF's Vertical Mining of Maximal Sequential Patterns (VMSP) algorithm and T-Rule Growth algorithm to efficiently extract and visualize movement patterns and behavioural rules of marine animals. The system also incorporates automated pre-processing to clean and prepare large datasets, reducing the time and effort required for manual handling. The visualization presents oceanic data from detection stations, allowing researchers to discover patterns in the movements of marine species. Our interface also provides geographical visualization of selected patterns or rules and helps users to navigate and find the relevant information. Key principles of our model include multi-view visualizations that provide comprehensive insights by displaying data from various perspectives simultaneously, and geo-spatial time-varying data visualization that captures and represents the dynamic movements of marine species. This approach allows domain experts to interactively explore and analyse sequential patterns and rules, enhancing their ability to make informed decisions. Additionally, feedback received, and insights gained from specialized professionals was used to determine if this approach would be a beneficial complement to existing practice.

Rule Extraction in Trained Feedforward Deep Neural Networks

Explainability is a key aspect of machine learning, necessary for ensuring transparency and trust in decision-making processes. As machine learning models become more complex, the integration of neural and symbolic approaches has emerged as a promising solution to the explainability problem. One effective solution involves using search techniques to extract rules from trained deep neural networks by examining weight and bias values and calculating their correlation with outputs. This article proposes incorporating cosine similarity in this process to narrow down the search space and identify the critical path connecting inputs to final results. Additionally, the integration of first-order logic (FOL) is suggested to provide a more comprehensive and interpretable understanding of the decision-making process. By leveraging cosine similarity and FOL, an innovative algorithm capable of extracting and explaining rule patterns learned by a feedforward trained neural network was developed and tested in two use cases, demonstrating its effectiveness in providing insights into model behavior.

Integration of Price Data in Wind Power Sales Systems Using Association Rule Mining Combined with Deep Learning Algorithms

With the growing attention of the global population towards renewable energy sources such as wind power derived from natural resources such as Solar, Geothermal, and Wind energy, there is a growing need for genuine estimation of wind energy. Wind energy is important in the supply of electricity in the global energy markets because of its enormous importance in the delivery of renewable energy. As such, it is crucial for accurate appreciation of the power of the wind to ably respond to issues that relate to the trading of power while at the same time addressing issues to do with planning, scheduling and strategic positioning of wind power generation. Therefore, the present work aims to develop a new model known as the Association Rule with DL-based Wind Power Generation and Price Prediction (ARDL-WPGPP). It utilizes two datasets: An Energy dataset includes various columns like tie, wind onshore forecast, and price actual, whereas the weather features dataset includes only wind speed. These datasets are combined to create a dataset where each transaction represents 2 hours of wind generation. A data mining approach is employed to uncover hidden patterns, rules, concepts, and correlations within these datasets, operating on various types of data including quantitative, textual, and multimedia formats. To efficiently extract rules from the dataset, an improved Apriori algorithm is introduced. Subsequently, the generated rules incorporating wind speed are passed into an improved LSTM model, which learns by comparing the label data. The price actual value serves as the label data, assigning labels to data points based on their actual price value. High price actual values indicate high wind power prices, while low values indicate low wind power prices.

Self-supervised representations and node embedding graph neural networks for accurate and multi-scale analysis of materials

Supervised machine learning algorithms, such as graph neural networks (GNN), have successfully predicted material properties. However, the superior performance of GNN usually relies on end-to-end learning on large material datasets, which may lose the physical insight of multi-scale information about materials. And the process of labeling data consumes many resources and inevitably introduces errors, which constrains the accuracy of prediction. We propose to train the GNN model by self-supervised learning on the node and edge information of the crystal graph. Compared with the popular manually constructed material descriptors, the self-supervised atomic representation can reach better prediction performance on material properties. Furthermore, it may provide physical insights by tuning the range information. Applying the self-supervised atomic representation on the magnetic moment datasets, we show how they can extract rules and information from the magnetic materials. To incorporate rich physical information into the GNN model, we develop the node embedding graph neural networks (NEGNN) framework and show significant improvements in the prediction performance. The self-supervised material representation and the NEGNN framework may investigate in-depth information from materials and can be applied to small datasets with increased prediction accuracy.

Sensory intelligence for extraction of abstract auditory rules from a speech sound stream in children with cochlear implants

ObjectiveSensory intelligence in the brain helps listeners automatically extract abstract auditory rules formed by invariant acoustic features from complex speech sound streams, presumably serving as the neural basis for speech comprehension. However, whether this intelligence is deficient in children with cochlear implants (CIs) remains unclear. MethodsMandarin Chinese monosyllables shared a flat lexical tone contour to form an abstract auditory rule but differed in other acoustic features to construct a complex speech sound stream. The abstract rule was occasionally violated by monosyllables with a rising or falling lexical tone. ResultsIn normal hearing (NH) children, the abstract auditory rule could be extracted, as revealed by a mismatch negativity (MMN) and a late discriminative negativity (LDN). However, the MMN and LDN were only evoked in CI children with good hearing and speech performance. NH children with a higher speech perception or spatial hearing score had a greater MMN. The LDN was attenuated with increasing age in NH children. ConclusionsThe sensory intelligence for extraction of auditory abstract rules, associated with speech perception, is deficient in CI children. This intelligence may gradually develop during childhood and adolescence. SignificanceDeficient sensory intelligence in CI children may aid in understanding poor speech comprehension in complex environments.

Real-Time Energy Management Strategy for Fuel Cell Vehicles Based on DP and Rule Extraction

Energy management strategy (EMS), as a core technology in fuel cell vehicles (FCVs), profoundly influences the lifespan of fuel cells and the economy of the vehicle. Aiming at the problem of the EMS of FCVs based on a global optimization algorithm not being applicable in real-time, a rule extraction-based EMS is proposed for fuel cell commercial vehicles. Based on the results of the dynamic programming (DP) algorithm in the CLTC-C cycle, the deep learning approach is employed to extract output power rules for fuel cell, leading to the establishment of a rule library. Using this library, a real-time applicable rule-based EMS is designed. The simulated driving platform is built in a CARLA, SUMO, and MATLAB/Simulink joint simulation environment. Simulation results indicate that the proposed strategy yields savings ranging from 3.64% to 8.96% in total costs when compared to the state machine-based strategy.

Exploring the 3-dimensional variability of websites' user-stories using triadic concept analysis

Configurable software systems and families of similar software systems are increasingly being considered by industry to provide software tailored to each customer's needs. Their development requires managing software variability, i.e. commonalities, differences and constraints. A primary step is properly analyzing the variability of software, which can be done at various levels, from specification to deployment. In this paper, we focus on the software variability expressed through user-stories, viz. short formatted sentences indicating which user role can perform which action at the specification level. At this level, variability is usually analyzed in a two dimension view, i.e. software described by features, and considering the roles apart. The novelty of this work is to model the three dimensions of the variability (i.e. software, roles, features) and explore it using Triadic Concept Analysis (TCA), an extension of Formal Concept Analysis. The variability exploration is based on the extraction of 3-dimensional implication rules. The adopted methodology is applied to a case study made of 65 commercial web sites in four domains, i.e. manga, martial arts sports equipment, board games including trading cards, and video-games. This work highlights the diversity of information provided by such methodology to draw directions for the development of a new product or for building software variability models.

A fuzzy rule extraction method based on Dempster–Shafer theory

A fuzzy rule extraction method based on Dempster–Shafer theory