Decision Tree Learning Techniques Research Articles

Relational Query Synthesis ⋈ Decision Tree Learning

We study the problem of synthesizing a core fragment of relational queries called select-project-join (SPJ) queries from input-output examples. Search-based synthesis techniques are suited to synthesizing projections and joins by navigating the network of relational tables but require additional supervision for synthesizing comparison predicates. On the other hand, decision tree learning techniques are suited to synthesizing comparison predicates when the input database can be summarized as a single labelled relational table. In this paper, we adapt and interleave methods from the domains of relational query synthesis and decision tree learning, and present an end-to-end framework for synthesizing relational queries with categorical and numerical comparison predicates. Our technique guarantees the completeness of the synthesis procedure and strongly encourages minimality of the synthesized program. We present Libra, an implementation of this technique and evaluate it on a benchmark suite of 1,475 instances of queries over 159 databases with multiple tables. Libra solves 1,361 of these instances in an average of 59 seconds per instance. It outperforms state-of-the-art program synthesis tools Scythe and PatSQL in terms of both the running time and the quality of the synthesized programs.

Learning Behavior Trees From Planning Experts Using Decision Tree and Logic Factorization

The increased popularity of Behavior Trees (BTs) in different fields of robotics requires efficient methods for learning BTs from data instead of tediously handcrafting them. Recent research in learning from demonstration reported encouraging results that this paper extends, improves and generalizes to arbitrary planning domains. We propose BT-Factor as a new method for learning expert knowledge by representing it in a BT. Execution traces of previously manually designed plans are used to generate a BT employing a combination of decision tree learning and logic factorization techniques originating from circuit design. We test BT-Factor in an industrially-relevant simulation environment from a mining scenario and compare it against a state-of-the-art BT learning method. The results show that our method generates compact BTs easy to interpret, and capable to capture accurately the relations that are implicit in the training data.

MECInOT: a multi-access edge computing and industrial internet of things emulator for the modelling and study of cybersecurity threats

In recent years, the Industrial Internet of Things (IIoT) has grown rapidly, a fact that has led to an increase in the number of cyberattacks that target this environment and the technologies that it brings together. Unfortunately, when it comes to using tools for stopping such attacks, it can be noticed that there are inherent weaknesses in this paradigm, such as limitations in computational capacity, memory and network bandwidth. Under these circumstances, the solutions used until now in conventional scenarios cannot be directly adopted by the IIoT, and so it is necessary to develop and design new ones that can effectively tackle this problem. Furthermore, these new solutions must be tested in order to verify their performance and viability, which requires testing architectures that are compatible with newly introduced IIoT topologies. With the aim of addressing these issues, this work proposes MECInOT, which is an architecture based on openLEON and capable of generating test scenarios for the IIoT environment. The performance of this architecture is validated by creating an intelligent threat detector based on tree-based algorithms, such as decision tree, random forest and other machine learning techniques. Which allows us to generate an intelligent and to demonstrate, we could generate an intelligent threat detector and demonstrate the suitability of our architecture for testing solutions in IIoT environments. In addition, by using MECInOT, we compare the performance of the different machine learning algorithms in an IIoT network. Firstly, we present the benefits of our proposal, and secondly, we describe the emulation of an IIoT environment while ensuring the repeatability of the experiments.

Comparison of robust machine-learning and deep-learning models for midterm electrical load forecasting

Electrical load forecasting (ELF) is gaining importance especially due to the severe impact of climate change on electrical energy usage and dynamically evolving smart grid technologies in the last decades. In this regard, medium-term load forecasting, a crucial need for power system planning (generation optimization and outages plan) and operation control, has become prominent in particular. Machine learning and deep learning-based techniques are currently trending approaches in electrical load estimation due to their capability to model complex non-linearity, feature abstraction and high accuracy, especially in the smart power systems environment.
 In this study, several load forecasting models based on machine learning methods which comprise linear regression (LR), decision tree (DT), random forest (RF), gradient boosting, adaBoost, and deep learning techniques such as recurrent neural network (RNN) and long short-term memory (LSTM) are studied for medium-term electrical load demand forecasting at an aggregated level. Performance metric results of these analyzes are presented in detail. State-of-the-art feature selection models are examined on the dataset and their effects on these forecasting methods are evaluated. Numerical results show that forecasting performance can be significantly improved. These results are validated by the results of other studies on the subject and found to be superior.

Classification of process conditions in martensitic stainless steel: A machine learning approach on magnetic Barkhausen emission signals

Sorting of materials at different stages of manufacturing processes is very important to ensure the final product quality. In this paper, the machine learning algorithms are used on the magnetic Barkhausen emission (MBE) signals for the classification of martensitic stainless-steel specimens, which are subjected to different heat treatments. The variation in MBE parameters, such as root mean square, peak height, peak position, and full width at half maximum, was found to be marginal for normalized and quenching and partitioned specimens even though their x-ray diffraction analysis showed distinct microstructural conditions. The conventional MBE parameters are used to train the machine learning models. The performance of decision tree and ensemble learning techniques, such as Bagging, Random Subspace, AdaBoost, RUSBoost, Total Boost, and LP Boost classifiers, is compared and found that the AdaBoost classifier provides the maximum accuracy of 98% in the classification of complex materials. The results showed that the machine learning approach using a limited number of features is sufficiently sensitive toward the classification of specimens that are otherwise indistinguishable in their conventional MBE response.

Design and Deployment of E-Health System Using Machine Learning in the Perspective of Developing Countries

Machine learning is tightening its grasp on many sectors of modern life and medical sector is not an exception. In developing countries like Bangladesh, disease classification process mostly remains manual, time consuming and sometimes erroneous. Designing an E-health system comprised of disease identification model would be a great aid in such circumstances. The automation of identifying the diseases with the help of machine learning will be more accurate and time-saving. In this paper, Decision Tree, Gaussian Naive-Bayes, Random Forest, Logistic Regression, k-NN, MLP, and SVM machine learning techniques are applied for three diseases: Dengue, Diabetes, and Thyroid. MLP for Dengue, Logistic Regression for Diabetes, and Random Forest for Thyroid performed the best with accuracies of 88.3%, 82.5%, and 98.5% respectively. Additionally, a medical specialist recommendation model and a medicine suggestion model are also integrated in the proposed E-Health system.

Predicting the Mechanical Properties of RCA-Based Concrete Using Supervised Machine Learning Algorithms.

Environment-friendly concrete is gaining popularity these days because it consumes less energy and causes less damage to the environment. Rapid increases in the population and demand for construction throughout the world lead to a significant deterioration or reduction in natural resources. Meanwhile, construction waste continues to grow at a high rate as older buildings are destroyed and demolished. As a result, the use of recycled materials may contribute to improving the quality of life and preventing environmental damage. Additionally, the application of recycled coarse aggregate (RCA) in concrete is essential for minimizing environmental issues. The compressive strength (CS) and splitting tensile strength (STS) of concrete containing RCA are predicted in this article using decision tree (DT) and AdaBoost machine learning (ML) techniques. A total of 344 data points with nine input variables (water, cement, fine aggregate, natural coarse aggregate, RCA, superplasticizers, water absorption of RCA and maximum size of RCA, density of RCA) were used to run the models. The data was validated using k-fold cross-validation and the coefficient correlation coefficient (R2), mean square error (MSE), mean absolute error (MAE), and root mean square error values (RMSE). However, the model’s performance was assessed using statistical checks. Additionally, sensitivity analysis was used to determine the impact of each variable on the forecasting of mechanical properties.

Sentiment Analysis using various Machine Learning and Deep Learning Techniques

Sentiment analysis has gained a lot of attention from researchers in the last year because it has been widely applied to a variety of application domains such as business, government, education, sports, tourism, biomedicine, and telecommunication services. Sentiment analysis is an automated computational method for studying or evaluating sentiments, feelings, and emotions expressed as comments, feedbacks, or critiques. The sentiment analysis process can be automated using machine learning techniques, which analyses text patterns faster. The supervised machine learning technique is the most used mechanism for sentiment analysis. The proposed work discusses the flow of sentiment analysis process and investigates the common supervised machine learning techniques such as multinomial naive bayes, Bernoulli naive bayes, logistic regression, support vector machine, random forest, K-nearest neighbor, decision tree, and deep learning techniques such as Long Short-Term Memory and Convolution Neural Network. The work examines such learning methods using standard data set and the experimental results of sentiment analysis demonstrate the performance of various classifiers taken in terms of the precision, recall, F1-score, RoC-Curve, accuracy, running time and k fold cross validation and helps in appreciating the novelty of the several deep learning techniques and also giving the user an overview of choosing the right technique for their application.

COVID-19 Risk Factors, Economic Factors, and Epidemiological Factors nexus on Economic Impact: Machine Learning and Structural Equation Modelling Approaches

Since the declaration of COVID-19 as a global pandemic, it has been transmitted to more than 200 nations of the world. The harmful impact of the pandemic on the economy of nations is far greater than anything suffered in almost a century. The main objective of this paper is to apply Structural Equation Modeling (SEM) and Machine Learning (ML) to determine the relationships among COVID-19 risk factors, epidemiology factors and economic factors. Structural equation modeling is a statistical technique for calculating and evaluating the relationships of manifest and latent variables. It explores the causal relationship between variables and at the same time taking measurement error into account. Bagging (BAG), Boosting (BST), Support Vector Machine (SVM), Decision Tree (DT) and Random Forest (RF) Machine Learning techniques was applied to predict the impact of COVID-19 risk factors. Data from patients who came into contact with coronavirus disease were collected from Kaggle database between 23 January 2020 and 24 June 2020. Results indicate that COVID-19 risk factors have negative effects on epidemiology factors. It also has negative effects on economic factors.

Default on a credit prediction using decision tree and ensemble learning techniques

Due to the high importance of credit risk management and the prediction of the default on a credit loan in banks in recent years in most developed countries. This research experimented with three machine learning algorithms which are decision tree and ensemble techniques based on a decision tree-like random forest which is bootstrap aggregation machine learning model and adaptive boosting is a boosting machine learning model. All models were used to perform a binary classification to classify the clients into trusted clients who would pay back the debts who are creditable and a none trusted clients who won’t pay the debts In the time which are who are not creditable. All the models gave nearly the same results based on the selected performance measures which are precision, recall, and f1 measure. There was no significant change when they performed on a real class imbalanced dataset of the default credit card clients of Taiwan.

Decision making strategy for detecting authenticated recommendations and identification of valuable customers in online shopping sites

The online shopping sites help customers not only choose various products but also place their reviews on those products. Their choices and opinions are often beneficial to other customers who want to buy the product. In this paper, we propose a method to identify only the best recommenders and the best buyers to validate the authenticity of the opinions and they are denoted as the valuable customers. The number of customers being influenced by the recommender develops a link or connection, this leads to a chain of connectivity. This concept has been justified by applying graph theory notion. Since the online sites are dynamic in nature, hence to keep the record set updated with valuable customers' information decision tree learning technique has been implemented. The effectiveness of the method is tested by applying it to a data set obtained from Amazon.

Classification and change detection of Sabah mangrove forest using decision-tree learning technique

The objective of this study is to determine the potential of decision tree-learning technique to classify and detects the changes of the Sabah mangrove forest area. The study area was conducted in the Mengkabong mangrove forest which is located on the west coast of Sabah. The multi-temporal of Landsat series (TM, ETM+, and OLI_TIRS) with five years interval data from 1990 and 2013 were used in this study. The results show that the use of decision-tree learning technique integrated with multi-temporal Landsat series and GIS data can be effective in delineating spatial and temporal change of the Sabah mangrove forest. The selection of suitable attributes from spectral features of Landsat data, topographic data and GIS database has promoted the high accuracy of the mangrove classification result with 90.8%. 40 hectares of Mengkabong mangrove were reduced from 1990 to 2013 and the fragmentation was obvious. In conclusion, the decision-tree learning technique was successfully classified and detects the changes of mangrove forest in the Mengkabong area.

Optimising pin-in-paste technology using gradient boosted decision trees

PurposeThis paper aims to present a robust prediction method for estimating the quality of electronic products assembled with pin-in-paste soldering technology. A specific board quality factor was also defined which describes the expected yield of the board assembly.Design/methodology/approachExperiments were performed to obtain the required input data for developing a prediction method based on decision tree learning techniques. A Type 4 lead-free solder paste (particle size 20–38 µm) was deposited by stencil printing with different printing speeds (from 20 mm/s to 70 mm/s) into the through-holes (0.8 mm, 1 mm, 1.1 mm, 1.4 mm) of an FR4 board. Hole-filling was investigated with X-ray analyses. Three test cases were evaluated.FindingsThe optimal parameters of the algorithm were determined as: subsample is 0.5, learning rate is 0.001, maximum tree depth is 6 and boosting iteration is 10,000. The mean absolute error, root mean square error and mean absolute percentage error resulted in 0.024, 0.03 and 3.5, respectively, on average for the prediction of the hole-filling value, based on the printing speed and hole-diameter after optimisation. Our method is able to predict the hole-filling in pin-in-paste technology for different through-hole diameters.Originality/valueNo research works are available in current literature regarding machine learning techniques for pin-in-paste technology. Therefore, we decided to develop a method using decision tree learning techniques for supporting the design of the stencil printing process for through-hole components and pin-in-paste technology. The first pass yield of the assembly can be enhanced, and the reflow soldering failures of pin-in-paste technology can be significantly reduced.

Interactive Visual Decision Tree for Developing Detection Rules of Attacks on Web Applications

Creating detection rules of attacks on web applications is not a trivial task, especially when the attacks are launched by experienced hackers. In such a situation, human expertise is essential to produce effective results. However, human users are easily overloaded by the huge input data, which is meant to be analyzed, learned from, and used to develop appropriate detection rules. To support human users in dealing with the information overload problem while developing detection rules of web application attacks, we propose a novel technique and tool called Interactive Visual Decision Tree (IVDT). IVDT is a variant of the popular decision tree learning technique introduced in research fields such as machine learning and data mining, with two additionally important features: visually supported data analysis and user-guided tree growing. Visually supported data analysis helps human users cope with high volume of training data while analyzing each node in the tree being built. On the other hand, user-guided tree growing allows human users to apply their own expertise and experience to create custom split condition for each tree node. A prototype implementation of IVDT is built and experimented to evaluate its effectiveness in terms of detection accuracy achieved by its users as well as ease of working with. The experiment results prove some advantages of IVDT over traditional decision tree learning method, but also point out its problems that should be handled in future improvements.

Discovering Process Model from Event Logs by Considering Overlapping Rules

Process Mining is a technique to automatically discover and analyze business processes from event logs. Discovering concurrent activities often uses process mining since there are many of them contained in business processes. Since researchers and practitioners are giving attention to the process discovery (one of process mining techniques), then the best result of the discovered process models is a must. Nowadays, using process execution data in the past, process models with rules underlying decisions in processes can be enriched, called decision mining. Rules defined over process data specify choices between multiple activities. One out of multiple activities is allowed to be executed in existing decision mining methods or it is known as mutually-exclusive rules. Not only mutually-exclusive rules, but also fully deterministic because all factors which influence decisions are recorded. However, because of non-determinism or incomplete information, there are some cases that are overlapping in process model. Moreover, the rules which are generated from existing method are not suitable with the recorded data. In this paper, a discovery technique for process model with data by considering the overlapping rules from event logs is presented. Discovering overlapping rules uses decision tree learning techniques, which fit the recorded data better than the existing method. Process model discovery from event logs is generated using Modified Time-Based Heuristics Miner Algorithm. Last, online book store management process model is presented in High-level BPMN Process Model.

Face Recognition, Musical Appraisal, and Emotional Crossmodal Bias.

Recent research on the crossmodal integration of visual and auditory perception suggests that evaluations of emotional information in one sensory modality may tend toward the emotional value generated in another sensory modality. This implies that the emotions elicited by musical stimuli can influence the perception of emotional stimuli presented in other sensory modalities, through a top-down process. The aim of this work was to investigate how crossmodal perceptual processing influences emotional face recognition and how potential modulation of this processing induced by music could be influenced by the subject's musical competence. We investigated how emotional face recognition processing could be modulated by listening to music and how this modulation varies according to the subjective emotional salience of the music and the listener's musical competence. The sample consisted of 24 participants: 12 professional musicians and 12 university students (non-musicians). Participants performed an emotional go/no-go task whilst listening to music by Albeniz, Chopin, or Mozart. The target stimuli were emotionally neutral facial expressions. We examined the N170 Event-Related Potential (ERP) and behavioral responses (i.e., motor reaction time to target recognition and musical emotional judgment). A linear mixed-effects model and a decision-tree learning technique were applied to N170 amplitudes and latencies. The main findings of the study were that musicians' behavioral responses and N170 is more affected by the emotional value of music administered in the emotional go/no-go task and this bias is also apparent in responses to the non-target emotional face. This suggests that emotional information, coming from multiple sensory channels, activates a crossmodal integration process that depends upon the stimuli emotional salience and the listener's appraisal.

Synchronization-Core-Based Discovery of Processes with Decomposable Cyclic Dependencies

Traditional process discovery techniques mine process models based upon event traces giving little consideration to workflow relevant data recorded in event logs. The neglect of such information usually leads to incorrect discovered models, especially when activities have decomposable cyclic dependencies. To address this problem, the recorded workflow relevant data and decision tree learning technique are utilized to classify cases into case clusters. Each case cluster contains causality and concurrency activity dependencies only. Then, a set of activity ordering relations are derived based on case clusters. And a synchronization-core-based process model is discovered from the ordering relations and composite cases. Finally, the discovered model is transformed to a BPMN model. The proposed approach is validated with a medical treatment process and an open event log. Meanwhile, a prototype system is presented.

Polarimetric SAR decomposition parameter subset selection and their optimal dynamic range evaluation for urban area classification using Random Forest

Urban area classification is important for monitoring the ever increasing urbanization and studying its environmental impact. Two NASA JPL's UAVSAR datasets of L-band (wavelength: 23cm) were used in this study for urban area classification. The two datasets used in this study are different in terms of urban area structures, building patterns, their geometric shapes and sizes. In these datasets, some urban areas appear oriented about the radar line of sight (LOS) while some areas appear non-oriented. In this study, roll invariant polarimetric SAR decomposition parameters were used to classify these urban areas.Random Forest (RF), which is an ensemble decision tree learning technique, was used in this study. RF performs parameter subset selection as a part of its classification procedure. In this study, parameter subsets were obtained and analyzed to infer scattering mechanisms useful for urban area classification. The Cloude–Pottier α, the Touzi dominant scattering amplitude αs1 and the anisotropy A were among the top six important parameters selected for both the datasets. However, it was observed that these parameters were ranked differently for the two datasets. The urban area classification using RF was compared with the Support Vector Machine (SVM) and the Maximum Likelihood Classifier (MLC) for both the datasets. RF outperforms SVM by 4% and MLC by 12% in Dataset 1. It also outperforms SVM and MLC by 3.5% and 11% respectively in Dataset 2.

의사결정나무를 이용한 다변량 공정관리 절차

현대의 제조공정은 컴퓨터의 발전과 통신 및 네트워크의 발달로 컴퓨터통합제조가 가능해졌다. 이로 인해 고품질 제품의 고속 생산공정이 확대되고, 공정에서 실시간으로 전송되는 다양한 품질변수들의 데이터 축적 또한 가능하게 되었다. 이를 관리하기 위해서는 다변량 통계적 공정관리 절차가 필요하다. 전통적으로 사용하는 다변량 관리도는 이상상태 발생시 이상신호를 주지만, 이상원인이 어떠한 변수에 어떠한 영향을 주는지에 대한 정보를 제공하지 않는다는 단점이 있다. 이를 보완하기 위해 데이터마이닝과 기계학습 기법을 이용할 수 있다. 이 논문에서는 의사결정나무 학습 기법을 이용한 다변량 공정관리 절차를 소개하고, 이변량인 경우 모의실험을 통하여 그 효율을 살펴보았다. 모의실험 결과를 살펴볼 때, 상관계수에 따라 이상상태 탐지 능력은 비슷한 것으로 나타났고, 이상상태에 대한 분류 정확도는 상관계수와 이상원인의 형태에 따라 차이가 있지만 기존의 다변량 관리도에서는 제공하지 않는 이상원인의 정보를 제공하는 장점이 있음을 알 수 있다. In today's manufacturing environment, the process data can be easily measured and transferred to a computer for analysis in a real-time mode. As a result, it is possible to monitor several correlated quality variables simultaneously. Various multivariate statistical process control (MSPC) procedures have been presented to detect an out-of-control event. Although the classical MSPC procedures give the out-of-control signal, it is difficult to determine which variable has caused the signal. In order to solve this problem, data mining and machine learning techniques can be considered. In this paper, we applied the technique of decision tree learning to the MSPC, and we did simulation for MSPC procedures to monitor the bivariate normal process means. The results of simulation show that the overall performance of the MSPC procedure using decision tree learning technique is similar for several values of correlation coefficient, and the accurate classification rates for out-of-control are different depending on the values of correlation coefficient and the shift magnitude. The introduced procedure has the advantage that it provides the information about assignable causes, which can be required by practitioners.

Improving the Performance of a Proxy Cache Using Very Fast Decision Tree Classifier

Abstract In this paper, we improved the performance of Web proxy cache replacement policies such as LRU and GDSF by adapting a Very Fast Decision Tree learning technique. In the first part, a sliding window method integrated with Very Fast Decision tree classifier (VFDT) to classify the web log data and predict the classes of web objects to be revisited again in future or not. In the second part, a Very Fast Decision Tree classifier is incorporated with proxy caching policies to form novel approaches known as VFDT-LRU and VFDT-GDSF. This proposed approach improves the performances of LRU and GDSF in terms of hit and byte hit ratio respectively.