Tree Rules Research Articles

Inherently interpretable machine learning for credit scoring: Optimal classification tree with hyperplane splits

An accurate and interpretable credit scoring model plays a crucial role in helping financial institutions reduce losses by promptly detecting, containing, and preventing defaulters. However, existing models often face a trade-off between interpretability and predictive accuracy. Traditional models like Logistic Regression (LR) offer high interpretability but may have limited predictive performance, while more complex models may improve accuracy at the expense of interpretability. In this paper, we tackle the credit scoring problem with imbalanced data by proposing two new classification models based on the optimal classification tree with hyperplane splits (OCT-H). OCT-H provides transparency and easy interpretation with ‘if-then’ decision tree rules. The first model, the cost-sensitive optimal classification tree with hyperplane splits (CSOCT-H). The second model, the optimal classification tree with hyperplane splits based on maximizing F1-Score (OCT-H-F1), aims to directly maximize the F1-score. To enhance model scalability, we introduce a data sample reduction method using data binning and feature selection. We then propose two solution methods: a heuristic approach and a method utilizing warm-start techniques to accelerate the solving process. We evaluated the proposed models on four public datasets. The results show that OCT-H significantly outperforms traditional interpretable models, such as Decision Trees (DT) and Logistic Regression (LR), in both predictive performance and interpretability. On certain datasets, OCT-H performs as well as or better than advanced ensemble tree models, effectively narrowing the gap between interpretable models and black-box models.

Advancing Animal Health: A Web-Based Expert System Utilizing Forward Chaining for Disease Diagnosis

The increasing prevalence of animal diseases, along with the increasing need for animal products, highlights the urgent need for efficient diagnostic tools in veterinary medicine. The aim of this research is to create an expert system that uses a forward chaining algorithm to diagnose animal diseases. The forward chaining algorithm is a deductive reasoning approach that starts with existing facts and uses expert tree rules for hypotheses. This process continues until the desired goal is achieved or no additional conclusions can be drawn. Even though there are developments in expert systems, there are still shortcomings in implementing the forward chain for rapid and precise diagnosis of livestock diseases. This work aims to fill this gap by developing an expert system that improves the accuracy and efficiency of disease diagnosis in the livestock industry. A database of animal diseases and symptoms was created by observing and interacting directly with farmers. The system architecture is specifically intended to optimize data processing and user engagement, enabling rapid diagnosis and treatment recommendations. This test shows a level of accuracy and precision, thereby reducing the possibility of misdiagnosis. The capacity of expert systems to provide fast and reliable diagnoses has the potential to improve livestock health management, thereby helping farmers maintain animal welfare and productivity. The results of this work advance the field of veterinary diagnostics and propose other uses of expert systems in animal health management.

Determining the minimum data size for the development of artificial neural network-based prediction models for rice pests in Korea

Sudden outbreaks of crop pests (insect pests and diseases) are increasing in Korea due to climate change and globalization. To prevent such outbreaks, it is necessary to predict and control pest occurrences in advance. Crop pests have been predicted through process-based or statistical modeling; however, the limitations of these models, which rely solely on historically acquired domain knowledge and data, have become increasingly prominent owing to climate change and rapidly changing agricultural ecosystems. To overcome these limitations, artificial neural network (ANN)-based models that use continuous pest survey data in the field have been investigated over the last decade. However, because pest survey data are collected by humans through process-mediated methods, fundamental problems exist in terms of data quality and size that may hinder the performance of the resulting ANN-based models. In this study, to determine the minimum pest data size required to ensure the optimal performance of ANN-based models, we developed feed-forward neural network models for 19 rice pests using 23 pest datasets collected from 149 districts by the Rural Development Administration of Korea over 19 years (2002–2020). Using each ANN-based model, the minimum data size required for the highest model performance achieved in this study was determined for all 19 rice pests. Furthermore, we developed a decision-tree rule to estimate the minimum data size based on the selected characteristics of each pest. The final Decision tree rule, based on the distinction between diseases and insect pests and the balance of pest data (the relative ratio of pest occurrence data to non-occurrence data), showed a relatively good performance (70.24 %) in the 3-fold cross-validation test. Overall, these results indicate that the minimum data sizes required for ANN modeling vary among rice pests, depending on the pest data characteristics, as indicated by the Decision tree rule developed in this study.

Implementation of Data Mining to Predict Student Study Period with Decision Tree Algorithm (C4.5)

Graduating on time is what every student wants to accomplish in college. Students of Prof. Dr. Hamka Muhammadiyah University are one of those who have this dream. Based on 2020 graduates data from the Tracer Study, 60% said the university had a high enough impact on improving competence. This data indicates that university needs to evaluate improvement of academic quality. Often, students have difficulty finding information about important factors that support achieving timely graduation. A prediction analysis is needed to provide information about the student's graduation study period. For this analysis, data mining is implemented using the classification function of the decision tree (C4.5) algorithm with RapidMiner tools. The methodology for implementing data mining follows the stages of Knowledge Discovery In Database (KDD), beginning with data collection, preprocessing, transformation, data mining, and evaluation. The research findings consist of visualization and decision tree rules that reveal GPA as the most influential factor in determining a student's study period.There is other information, namely, students graduated on time (less than equal to 4 years) amounted to 170 or 54.5% and students did not graduate on time (more than 4 years) amounted to 142 or 45.6%. Testing the performance of decision tree (C4.5) utilizing confusion matrix through RapidMiner tools, resulted in accuracy reaching 83.87%, with precision of 87.50% and recall of 91.18%. Provides evidence that the decision tree algorithm (C4.5) has optimal performance to provide valuable information about predicting student graduation in order to increase student enrollment with the right study period.

Identification of the Yield Rate by a Hybrid Fuzzy Control PID-Based Four-Stage Model: A Case Study of Optical Filter Industry

With the vigorous development of emerging technology and the advent of the Internet generation, high-speed Internet and fast transmission 5G wireless networks contribute to interpersonal communication. Now, the Internet has become popular and widely available, and human life is inseparable from various experiences on the Internet. Many base stations and data centers have been established to convert and switch from electrical transmission to optical transmission; thus, it is entering the new era of optical fiber networks and optical communication technologies. For optical communication, the manufacturing of components for the purpose of high-speed networks is a key process, and the requirement for the stability of its production conditions is very strict. In particular, product yields are always low due to the restriction of high-precision specifications associated with the limitations of too many factors. Given these reasons, this study proposes a hybrid fuzzy control-based model for industry data applications to organize advanced techniques of box-and-whisker plot method, association rule, and decision trees to find out the determinants that affect the yield rate of products and then use the fuzzy control Proportional-Integral-Derivative (PID) method to manage the determinants. Since it is unrealistic to test the real machine online operation at the manufacturing stage, the simulation software supersedes this for improved results, and a mathematical neural network is used to verify the given data to confirm whether its result is similar to that of the simulation. The study suggests that excessive temperature differentials between substrate and cavity can lead to low yields. It suggests using fuzzy control technology for temperature management, which could increase yield, reduce labor costs, and accelerate the transition to high-speed networks by mass-producing high-precision optical filters.

ПРИМЕНЕНИЕ МЕТОДА РЕШАЮЩИХ ДЕРЕВЬЕВ ДЛЯ РЕШЕНИЯ ЗАДАЧ АВТОМАТИЧЕСКОГО УЧЕТА РАСХОДА И КОЛИЧЕСТВА ВЛАЖНОГО ГАЗА

This article discusses the application of the decision tree method to solve problems of automatic accounting of the amount of natural gas. This method is used for classification and forecasting tasks. This method is also called a decision rule tree, classification tree, and regression tree. Within the framework of this work, the method of solving trees is used to effectively solve the prob-lem of automatic accounting of the consumption and quantity of wet natural gas. The article considers the algorithm, the process, the structure of the system, the structure, and also analyzes the results of the system.

A genomic rule-based KNN model for fast flux botnet detection

Fast Flux Botnet (FFB) is an advance method developed by cyber criminals to perpetrate distributed malicious attacks. The major problems of existing FFB detection systems are the vulnerability to evasion mechanisms, long detection time, and high dimensionality of the feature set. In this study, an improved FFB detection architecture called Bot-FFX was developed to address some of these problems. The developed Bot-FFX consists of four modules: extractor, filter, resolver, and detector. The extractor module is responsible for Domain Name System (DNS) queries on domains. The filter module can classify the incoming domains as either blacklist or whitelist and sends the unclassified domains to the resolver. The resolver extracts all IP addresses associated with the domain at its Time-To-Live (TTL) within a time frame of 10 min. The detector module uses a rule-based Genetic Algorithm (GA) and K-Nearest Neighbor (KNN) for botnet detection. The detector computed the Standard Deviation of Round Trip Time (SDRTT), Average Google Hits (AGH) and Genetic Threshold Value (GTV) for all IP addresses associated with the domains. The detector, built on a decision tree rules and the K-Dimensional (KD) tree KNN algorithm, classified the domains using the set of IP addresses, SDRTT, AGH, and GTV. The Bot-FFX was implemented on a dataset of 2,000 benign domains and 1,630 botnet domains. The dataset was split into 50% training and 50% testing sets. The evaluation results on the same datasets showed that Bot-FFX is an effective FFB detection system with accuracy, false positive, and false negative of 99.178%, 0.8%, and 0.8% respectively.

Towards a benchmark of national training requirements for continuous cover forestry (CCF) in Sweden

Continuous cover forestry (CCF) is forest management based on ecological and biological principles. CCF particularly requires the abandonment of clearfelling practices in favour of more natural approaches of regeneration. Recently, CCF has been identified as a way to mitigate climate change, to bring about forest conservation and to meet the diverse requirements of recreation forests. EU strategies support the use of CCF and Sweden is committed to the transformation of 20% of its plantation forest land to CCF. This policy change meets the Swedish forest industry rather unprepared. CCF training is therefore urgently required and we applied marteloscope techniques to begin with establishing a benchmark of training requirements. A marteloscope is a forest research plot where all trees are measured and have clearly visible numbers on the stem surface. We carried out a first marteloscope experiment at the Svartberget experimental forest near Umeå in Northern Sweden involving 13 test persons that we asked to carry out CCF management by selecting trees that are supposed to stay behind and others that are to be taken out in order to achieve CCF objectives. We applied specialised statistics to analyse the trainees’ choices and thus to measure their current state of silvicultural knowledge and experience. The results were interpreted in the context of data previously obtained from 26 comparable marteloscope experiments in Britain. The Svartberget results were in parts similar to and sometimes even closer to theoretical expectations than the British results, but also revealed that more intensive training was required in individual-based forest management, which is an important part of CCF. A new didactic technique, the competitor-for-frame tree rule, tightening the link between evicted and residual trees has contributed to the good tree-selection performance at Svartberget.

Embedded Gateway Security Detection Technology Based on the Deep Neural Network Rule Extraction

Aiming at the network security problem of power system cable trench control industrial Internet system, we studied an intrusion detection method applied to the embedded industrial Internet of Things gateway. This method extracts rules from the DBN-DNN deep neural network to obtain intrusion detection models that are conducive to integration into embedded systems. We first use the DBN network to reduce the dimensionality of the data, then use the DNN to train the classification model, and extract the rules from the DNN’s neurons to form a rule tree for intrusion detection. The KDD CUP99 training database is used to verify the feasibility of the method, and the test is carried out in the embedded gateway. The results show that the detection method based on rule extraction used in this paper can ensure detection efficiency and accuracy compared to the traditional detection methods. At the same time, it saves more computing resources and is more conducive to integration in embedded gateway systems.

A transparent machine learning algorithm to manage diabetes: TDMSML

Diabetes is nowadays a very common medical problem among the people worldwide. The disease is becoming more prevalent with the modern and hectic lifestyle followed by people. As a result, designing an adequate medical expert system to assist physicians in treating the disease on time is critical. Expert systems are required to identify the major cause(s) of the disease, so that precautionary measures can be taken ahead of time. Several medical expert systems have already been proposed, but each has its own set of shortcomings, such as the use of trial and error methods, trivial decision-making procedures, and so on. As a result, this paper proposes a Transparent Diabetes Management System Using Machine Learning (TDMSML) expert system that uses decision tree rules to identify the major factor(s) of diabetes. The TDMSML model comprises of three phases: rule generation, transparent rule selection, and major factor identification. The rule generation phase generates rules using decision tree. Transparent rule selection stage selects the transparent rules followed by pruning the redundant rules to get the minimized rule-set. The major factor identification stage extracts the major factor(s) with range(s) from the minimized rule-set. These factor(s) with certain range(s) are characterized as major cause(s) of diabetes disease. The model is validated with the Pima Indian diabetes data set collected from Kaggle.

Research on Automatic Generation of Engineering Geological Survey Report Paragraphs Based on Dynamic Data Template

As the final result of geological exploration, the engineering geological survey report needs to be prepared in large quantities, and the efficiency of manual preparation of the report is not high enough. To solve this problem, this paper proposes a standardized report paragraph generation model based on dynamic data templates. Through dynamic data template, the structure and content of report paragraphs are directly controlled by data; To solve the problem of template generating documents, a set of node tree rules is designed, and LaTeX files are generated by traversing the template node tree. The experiment shows that paragraphs can be modified by modifying data in terms of structure, content and text format, which improves work efficiency and has high use value in highly standardized engineering geological survey reports.

The Analysis of English Sentence Components Based on Decision Tree Classification Algorithm

Decision tree is an important classification method in data excavation technology. It is a predictive analysis model expressed in the form of a tree structure (including binary trees and poly trees). The decision tree method is a more general classification function approximation method. It is an algorithm commonly used in predictive models to find some potentially valuable information by purposefully classifying a large amount of data. In this article, the author tries to analyze the English sentence components based on the decision tree classification algorithm. The author starts with the decision tree, extracts the decision tree rules, and generates a classifier by effectively sorting the decision tree rules, and applies it to classification prediction.

Application of Advanced Hybrid Models to Identify the Sustainable Financial Management Clients of Long-Term Care Insurance Policy

The rapid growth of the aging population and the rate of disabled people with physical and mental disorders is increasing the demand for long-term care. The decline in family care could lead to social and economic collapse. In order to reduce the burden of long-term care, long-term care insurance has become one of the most competitive products in the life insurance industry. In the previous literature review, few scholars engaged in the research on this topic with data mining technology, which was motivated to trigger the formation of this study and hoped to increase the different aspects of academic research. The purpose of this study is to develop the long-term insurance business from the original list of insurance clients, to predict whether the sustainable financial management clients will buy the long-term care insurance policies, and to establish a feasible prediction model to assist life insurance companies. This study aims to establish the classified prediction models of Models I~X, to dismantle the data with the percentage split and 10-fold cross validation, plus the application of two kinds of technology as feature selection and data discretization, for the data mining of twenty-three kinds of algorithms in seven different categories (Bayes, Function, Lazy, Meta, Misc, Rule, and Decision Tree) through the data collected from the insurance company database, and to select 20 conditional attributes and 1 decisional attribute (whether to buy the long-term insurance policy or not). The decision attribute is binary classification method for empirical data analysis. The empirical results show that: (1) the marital status, total number of policies purchased, and total amount of policies (including long-term care insurance) are found to be the three important factors affecting the decision attribute; (2) the most stable models are the advanced hybrid Models V and X; and (3) the best classifier is Decision Tree J48 algorithm for the study data used.

Cybersecurity Knowledge Extraction Using XAI

Global networking, growing computer infrastructure complexity and the ongoing migration of many private and business aspects to the electronic domain commonly mandate using cutting-edge technologies based on data analysis, machine learning, and artificial intelligence to ensure high levels of network and information system security. Transparency is a major barrier to the deployment of black box intelligent systems in high-risk domains, such as the cybersecurity domain, with the problem getting worse as machine learning models increase in complexity. In this research, explainable machine learning is used to extract information from the CIC-IDS2017 dataset and to critically contrast the knowledge attained by analyzing if–then decision tree rules with the knowledge attained by the SHAP approach. The paper compares the challenges of the knowledge extraction using the SHAP method and the if–then decision tree rules, providing guidelines regarding different approaches suited to specific situations.

Bidirectional Flow Decision Tree for Reliable Remote Sensing Image Scene Classification

Remote sensing image scene classification (RSISC), which aims to classify scene categories for remote sensing imagery, has broad applications in various fields. Recent deep learning (DL) successes have led to a new wave of RSISC applications; however, they lack explainability and trustworthiness. Here, we propose a bidirectional flow decision tree (BFDT) module to create a reliable RS scene classification framework. Our algorithm combines BFDT and Convolutional Neural Networks (CNNs) to make the decision process easily interpretable. First, we extract multilevel feature information from the pretrained CNN model, which provides the basis for constructing the subsequent hierarchical structure. Then the model uses the discriminative nature of scene features at different levels to gradually refine similar subsets and learn the interclass hierarchy. Meanwhile, the last fully connected layer embeds decision rules for the decision tree from the bottom up. Finally, the cascading softmax loss is used to train and learn the depth features based on the hierarchical structure formed by the tree structure that contains rich remote sensing information. We also discovered that superclass results can be obtained well for unseen classes due to its unique tree structure hierarchical property, which results in our model having a good generalization effect. The experimental results align with theoretical predictions using three popular datasets. Our proposed framework provides explainable results, leading to correctable and trustworthy approaches.

Predicting students' performance in English and Mathematics using data mining techniques.

This study attempts to predict secondary school students’ performance in English and Mathematics subjects using data mining (DM) techniques. It aims to provide insights into predictors of students’ performance in English and Mathematics, characteristics of students with different levels of performance, the most effective DM technique for students’ performance prediction, and the relationship between these two subjects. The study employed the archival data of students who were 16 years old in 2019 and sat for the Malaysian Certificate of Examination (MCE) in 2021. The learning of English and Mathematics is a concern in many countries. Three main factors, namely students’ past academic performance, demographics, and psychological attributes were scrutinized to identify their impact on the prediction. This study utilized the Orange software for the DM process. It employed Decision Tree (DT) rules to determine the characteristics of students with low, moderate, and high performance in English and Mathematics subjects. DT and Naïve Bayes (NB) techniques show the best predictive performance for English and Mathematics subjects, respectively. Such characteristics and predictions may cue appropriate interventions to improve students’ performance in these subjects. This study revealed students’ past academic performance as the most critical predictor, as well as a few demographics and psychological attributes. By examining top predictors derived using four different classifier types, this study found that students’ past Mathematics performance predicts their MCE English performance and students’ past English performance predicts their MCE Mathematics performance. This finding shows students’ performances in both subjects are interrelated.

Contextualized Drug-Drug Interaction Management Improves Clinical Utility Compared With Basic Drug-Drug Interaction Management in Hospitalized Patients.

Drug–drug interactions (DDIs) frequently trigger adverse drug events or reduced efficacy. Most DDI alerts, however, are overridden because of irrelevance for the specific patient. Basic DDI clinical decision support (CDS) systems offer limited possibilities for decreasing the number of irrelevant DDI alerts without missing relevant ones. Computerized decision tree rules were designed to context‐dependently suppress irrelevant DDI alerts. A crossover study was performed to compare the clinical utility of contextualized and basic DDI management in hospitalized patients. First, a basic DDI‐CDS system was used in clinical practice while contextualized DDI alerts were collected in the background. Next, this process was reversed. All medication orders (MOs) from hospitalized patients with at least one DDI alert were included. The following outcome measures were used to assess clinical utility: positive predictive value (PPV), negative predictive value (NPV), number of pharmacy interventions (PIs)/1,000 MOs, and the median time spent on DDI management/1,000 MOs. During the basic DDI management phase 1,919 MOs/day were included, triggering 220 DDI alerts/1,000 MOs; showing 57 basic DDI alerts/1,000 MOs to pharmacy staff; PPV was 2.8% with 1.6 PIs/1,000 MOs costing 37.2 minutes/1,000 MOs. No DDIs were missed by the contextualized CDS system (NPV 100%). During the contextualized DDI management phase 1,853 MOs/day were included, triggering 244 basic DDI alerts/1,000 MOs, showing 9.6 contextualized DDIs/1,000 MOs to pharmacy staff; PPV was 41.4% (P < 0.01), with 4.0 PIs/1,000 MOs (P < 0.01) and 13.7 minutes/1,000 MOs. The clinical utility of contextualized DDI management exceeds that of basic DDI management.

Integration of Morphometrics and Machine Learning Enables Accurate Distinction between Wild and Farmed Common Carp.

Morphology and feature selection are key approaches to address several issues in fisheries science and stock management, such as the hypothesis of admixture of Caspian common carp (Cyprinus carpio) and farmed carp stocks in Iran. The present study was performed to investigate the population classification of common carp in the southern Caspian basin using data mining algorithms to find the most important characteristic(s) differing between Iranian and farmed common carp. A total of 74 individuals were collected from three locations within the southern Caspian basin and from one farm between November 2015 and April 2016. A dataset of 26 traditional morphometric (TMM) attributes and a dataset of 14 geometric landmark points were constructed and then subjected to various machine learning methods. In general, the machine learning methods had a higher prediction rate with TMM datasets. The highest decision tree accuracy of 77% was obtained by rule and decision tree parallel algorithms, and “head height on eye area” was selected as the best marker to distinguish between wild and farmed common carp. Various machine learning algorithms were evaluated, and we found that the linear discriminant was the best method, with 81.1% accuracy. The results obtained from this novel approach indicate that Darwin’s domestication syndrome is observed in common carp. Moreover, they pave the way for automated detection of farmed fish, which will be most beneficial to detect escapees and improve restocking programs.

Feature Extraction of National Physical Fitness Data Based on Data Mining

In order to better understand and optimize the national physical fitness, this paper puts forward the national physical fitness data change feature extraction method based on data mining, uses the decision tree and association rule data mining algorithm to collect the national physical fitness data in recent years, constructs the database to realize the effective data management, and uses the data mining algorithm to construct the physical fitness change feature evaluation index. Finally, through experiments, it is confirmed that the national physique data change feature extraction method based on data mining has high effectiveness in the process of practical application. It can better understand the national physique change trend and put forward targeted suggestions for national physique health optimization.

A Comparative Analysis of the Performance of Multiple Data Mining Classification Approaches Using the Kn Fold Validation

In Healthcare the data is very large and sensitive. The data is mandatory to be handled very carefully without any negligence. A variety of data mining categorization approaches have been employed in the healthcare industry to assess the quality of services. On the basis of 150 patients' records, this study provides and evaluates the experience of implementing various data mining methodologies and procedures. Using data mining techniques, a new method for determining a product's correctness has emerged. The evaluation of performance on data mining classification by using a different algorithms like Decision Tree, Naïve Bayes, KNN, Radom Tree Set and Rule Model. Finally we tend to aim to contemplate the performance analysis of accuracy, sensitivity and specificity proportion to produce a result.