Rule Induction Algorithm Research Articles

A new adaptive intrusion detection system based on the intersection of two different classifiers

Nowadays, the intrusion detection system (IDS) has become one of the most important weapons against cyber-attacks. The simple single-level IDS cannot detect both attack types and normal behaviour with high detection rate. To overcome this limit, we propose a new approach for intrusion detection. The idea of this paper is to use two different classifiers iteratively, where each-iteration represents one level in the built model. To ensure the adaptation of our model, we add a new level whenever the sum of new attacks and the rest of the training dataset reaches the threshold. To build our model, we have used Fuzzy Unordered Rule Induction Algorithm and Random Forests as classifiers. The experiment on the KDD99 dataset shows the high performance of our model that demonstrates its ability to detect the low frequent attack without losing their high performance in the detection of frequent attack and normal behaviour. Furthermore, our model gives the highest detection rate and the highest accuracy, compared with some models well known in the literature related to intrusion detection.

TSD based framework for mining the induction rules

Sex determination mainly encompasses two aspects: genotypic sex determination (GSD) and temperature-dependent sex determination (TSD). Genotypic sex determination performs its task by observing the presence of sex chromosomes. In many reptiles sex determination is greatly influenced by the environmental conditions such as temperature of the nest, weight and size of eggs. A nature inspired algorithm which mimics the mechanism of temperature dependent sex determination (TSD) has been introduced for mining the classification rules from datasets. A comparison of proposed TSD algorithm with other well known rule induction algorithms like PRISM, C4.5, 1-R, CN2, and NN has been evaluated on some bench mark datasets.

Fuzzy unordered rule induction algorithm in text categorization on top of geometric particle swarm optimization term selection

Rapid growth of digital information requires automated handling and organization of documents. The two main stages in automated document categorization are (i) term reduction and (ii) classificatio...

A knowledge-based intelligent framework for anterior cruciate ligament rehabilitation monitoring

This study presents an integration of knowledge-based system and intelligent methods to develop a recovery monitoring framework for post anterior cruciate ligament (ACL) injured/reconstructed subjects. The case based reasoning methodology has been combined with fuzzy clustering and intelligent classification techniques in order to develop a knowledge base and a learning model for identifying the recovery stage of ACL-reconstructed subjects and objectively monitoring the progress during the convalescence regimen. The system records kinematics and neuromuscular signals from lower limbs of healthy and ACL-reconstructed subjects using self adjusted non-invasive body-mounted wireless sensors. These bio-signals are synchronized and integrated, and a combined feature set is generated by performing data transformation using wavelet decomposition and feature reduction techniques. The knowledge base stores the subjects’ profiles, their recovery sessions’ data and problem/solution pairs for different activities monitored during the course of rehabilitation. Fuzzy clustering technique has been employed to form the initial groups of subjects at similar stage of recovery. In order to classify the recovery stage of subjects (i.e. retrieval of similar cases), adaptive neuro-fuzzy inference system (ANFIS), fuzzy unordered rule induction algorithm (FURIA) and support vector machine (SVM) have been applied and compared. The system has been successfully tested on a group of healthy and post-operated athletes for analyzing their performance in two activities (ambulation at various speeds and one leg balance testing) selected from the rehabilitation protocol. The case adaptation and retention is a semi-automatic process requiring input from the physiotherapists and physiatrists. This intelligent framework can be utilized by physiatrists, physiotherapists, sports trainers and clinicians for multiple purposes including maintaining athletes’ profile, monitoring progress of recovery, classifying recovery status, adapting recovery protocols and predicting/comparing athletes’ sports performance. Further, the knowledge base can easily be extended and enhanced for monitoring different types of sports activities.

An Efficient Machine Learning Based Classification Scheme for Detecting Distributed Command & Control Traffic of P2P Botnets

Biggest internet security threat is the rise of Botnets having modular and flexible structures. The combined power of thousands of remotely controlled computers increases the speed and severity of attacks. In this paper, we provide a comparative analysis of machine-learning based classification of botnet command & control(C&C) traffic for proactive detection of Peer-to- Peer (P2P) botnets. We combine some of selected botnet C&C traffic flow features with that of carefully selected botnet behavioral characteristic features for better classification using machine learning algorithms. Our simulation results show that our method is very effective having very good test accuracy and very little training time. We compare the performances of Decision Tree (C4.5), Bayesian Network and Linear Support Vector Machines using performance metrics like accuracy, sensitivity, positive predictive value(PPV) and F-Measure. We also provide a comparative analysis of our predictive models using AUC (area under ROC curve). Finally, we propose a rule induction algorithm from original C4.5 algorithm of Quinlan. Our proposed algorithm produces better accuracy than the original decision tree classifier.

Fuzzy unordered rule induction algorithm in text categorization on top of geometric particle swarm optimization term selection

Rapid growth of digital information requires automated handling and organization of documents. The two main stages in automated document categorization are (i) term reduction and (ii) classification. In this paper, we present a novel two-stage term reduction strategy based on Information Gain (IG) theory and Geometric Particle Swarm Optimization (GPSO) search. We evaluate performance of the proposed term reduction approach with use of a new classifier, fuzzy unordered rule induction algorithm (FURIA) to categorize multi-label texts. In order to evaluate the performance of FURIA quantitatively, we compared it against two widely used algorithms, Naïve Bayes and Support Vector Machine (SVM). Text Categorization (TC) performance of the proposed term reduction strategy is validated with use of Reuters-21578 and OHSUMED text collection datasets. The experimental results show that performance of the proposed term reduction method is efficient for document organization tasks.

Data-driven adaptive selection of rule quality measures for improving rule induction and filtration algorithms

This paper presents a proposal of a rule induction algorithm selecting a rule quality measure adaptively. The quality measure plays the role of an optimization criterion of the generated rules. Nine quality measures applied by the algorithm are presented and discussed in the paper. It is shown experimentally that the proposed algorithm provides us with obtaining a classifier of the best quality. During experiments, three criteria of the classifier quality were considered: overall accuracy, balanced accuracy (average accuracy of decision classes), and complexity of the classifier (understood to mean the number of induced rules). The experiments were carried out on 34 data sets coming from the UCI machine learning repository. Moreover, a proposal of four-rule filtration algorithms is presented in the paper. Their task is to limit the number of rules in the classifier. In particular, filtration influence on the classifier quality is studied.

Ischemic heart disease detection using selected machine learning methods

This article presents a method based on support vector machines (SVMs) and the Osuna–Platt algorithm used to diagnose the ischemic heart disease. It also includes the necessary concepts from the optimization theory which make it possible to formulate the problem for support vectors and the Osuna–Platt algorithm applied. Next, a principal component analysis (PCA) algorithm used is also presented. Heart images acquired using Single Photon Emission Computed Tomography (SPECT) have been used in the experimental part. Results of classifying cardiac SPECT images using SVM, PCA and neural networks are compared here with those obtained using another method of machine learning – CLIP3 – a combination of the decision tree algorithm and the rule induction algorithm. Tests against an SPECT image database have shown that SVMs are generally more accurate and specific, while the PCA algorithm is the most sensitive for all data sets analysed in this research project.

A novel algorithm for fuzzy rule induction in data mining

Rule induction as a method of constructing classifiers is of particular interest to data mining because it generates models in the form of If-Then rules which are more expressive and easier for humans to comprehend and check. Several induction algorithms have been developed to learn classification rules. However, most of these algorithms are based on ‘crisp’ data and produce ‘crisp’ models. This paper presents FuzzySRI, a novel algorithm based on the techniques of fuzzy sets and fuzzy logic for inducing fuzzy classification rules. The algorithm possesses the clear knowledge representation capability of rule induction methods and the ability of fuzzy techniques to handle vague information. Experimental results show that FuzzySRI can outperform other fuzzy and non-fuzzy learning systems in terms of predictive accuracy, comprehensibility, and computational efficiency. It is also shown that FuzzySRI can be successfully applied to an industrial application concerning the automatic identification of machine faults.

Enhanced Rule Induction Algorithm for Customer Relationship Management

Customer Relationship Management (CRM) helps businesses to gain insight into the behavior of customers and their value so that the company can increase their profit by acting according to the cu stomer characteristics. In order to analyze the customer needs and behavior, data mining is used to extract information from the customer database. For analyzing the customer behavior, the important attributes in the customer database are first chosen and then the y are segmented into groups using clustering algorithm based on those attribute values. The rules are then generated to describe the cus tomers in each group using LEM2 (Learning from Examples Module, version 2) algorithm and the proposed algorithm. These rules can be used by the business people to predict the behavior and to vary their promotional activities like coupon distribution and special disco unts. It is observed that the proposed algorithm is better in terms of time complexity and performance measures.

Fuzzy Unordered Rule Induction for evaluating cardiac Arrhythmia

Heart rate variations reveal current or impending heart/cardiac diseases. A non-stationary signal — heart rate — is measured using Electrocardiogram (ECG) to assess cardiac Arrhythmia. But studying ECG reports is both tedious and time consuming when you have to locate abnormalities from the collected data. This is overcome by computer based analytical tools for in-depth data study/classification for diagnosis. Automatic arrhythmia assessment is easy due to the existence of image processing techniques. Many algorithms exist for ECG signals detection/classification. This paper investigates RR interval based ECG classification procedures for arrhythmic beat classification, a process based on RR interval beat extraction using Symlet on ECG data. Extracted RR data is used as a classification feature with beats being classified through a boosting algorithm and Fuzzy Unordered Rule Induction Algorithm (FURIA). Classification efficiency evaluation was through the MITBIH arrhythmia database.

DATA MINING FOR KNOWLEDGE DISCOVERY FROM OBJECT-BASED SEGMENTATION OF VHR REMOTELY SENSED IMAGERY

Abstract. The success of the object-based image analysis (OBIA) paradigm can be attributed to the fact that regions obtained by means of segmentation process are depicted with a variety of spectral, shape, texture and context characteristics. These representative objectsattributes can be assigned to different land-cover/land-use types by means of two options. The first is to use supervised classifiers such as K-nearest neighbors (KNN) and Support Vector Machine (SVM), the second is to create classification rules. Supervised classifiers perform very well and have generally higher accuracies. However one of their drawbacks is that they provide no explicit knowledge in understandable and interpretable forms. The building of the rule set is generally based on the domain expert knowledge when dealing with a small number of classes and a small number of attributes, but having a dozens of continuously valued attributes attached to each image object makes it a tedious task and experts quickly get overwhelmed and become totally helpless. This is where data mining techniques for knowledge discovering help to understand the hidden relationships between classes and their attached attributes. The aim of this paper is to highlight the benefits of using knowledge discovery and data-mining tools, especially rule induction algorithms for useful and accurate information extraction from high spatial resolution remotely sensed imagery.

Keratin protein property based classification of mammals and non-mammals using machine learning techniques

Keratin protein is ubiquitous in most vertebrates and invertebrates, and has several important cellular and extracellular functions that are related to survival and protection. Keratin function has played a significant role in the natural selection of an organism. Hence, it acts as a marker of evolution. Much information about an organism and its evolution can therefore be obtained by investigating this important protein. In the present study, Keratin sequences were extracted from public data repositories and various important sequential, structural and physicochemical properties were computed and used for preparing the dataset. The dataset containing two classes, namely mammals (Class-1) and non-mammals (Class-0), was prepared, and rigorous classification analysis was performed. To reduce the complexity of the dataset containing 56 parameters and to achieve improved accuracy, feature selection was done using the t-statistic. The 20 best features (parameters) were selected for further classification analysis using computational algorithms which included SVM, KNN, Neural Network, Logistic regression, Meta-modeling, Tree Induction, Rule Induction, Discriminant analysis and Bayesian Modeling. Statistical methods were used to evaluate the output. Logistic regression was found to be the most effective algorithm for classification, with greater than 96% accuracy using a 10-fold cross validation analysis. KNN, SVM and Rule Induction algorithms also were found to be efficacious for classification.

Exploratory analysis of methods for automated classification of laboratory test orders into syndromic groups in veterinary medicine.

BackgroundRecent focus on earlier detection of pathogen introduction in human and animal populations has led to the development of surveillance systems based on automated monitoring of health data. Real- or near real-time monitoring of pre-diagnostic data requires automated classification of records into syndromes–syndromic surveillance–using algorithms that incorporate medical knowledge in a reliable and efficient way, while remaining comprehensible to end users.MethodsThis paper describes the application of two of machine learning (Naïve Bayes and Decision Trees) and rule-based methods to extract syndromic information from laboratory test requests submitted to a veterinary diagnostic laboratory.ResultsHigh performance (F1-macro = 0.9995) was achieved through the use of a rule-based syndrome classifier, based on rule induction followed by manual modification during the construction phase, which also resulted in clear interpretability of the resulting classification process. An unmodified rule induction algorithm achieved an F1-micro score of 0.979 though this fell to 0.677 when performance for individual classes was averaged in an unweighted manner (F1-macro), due to the fact that the algorithm failed to learn 3 of the 16 classes from the training set. Decision Trees showed equal interpretability to the rule-based approaches, but achieved an F1-micro score of 0.923 (falling to 0.311 when classes are given equal weight). A Naïve Bayes classifier learned all classes and achieved high performance (F1-micro = 0.994 and F1-macro = .955), however the classification process is not transparent to the domain experts.ConclusionThe use of a manually customised rule set allowed for the development of a system for classification of laboratory tests into syndromic groups with very high performance, and high interpretability by the domain experts. Further research is required to develop internal validation rules in order to establish automated methods to update model rules without user input.

Knowledge Acquisition and Management for Marine Diesel Engine Diagnosis

The paper presents the research relevant to knowledge acquisition for marine diesel engine diagnosis. Basic sources of knowledge which can be used for construction of diagnostics knowledge base are identified. The basic knowledge related to the diesel diagnostic was undertaken from experts and from diagnostic database. The paper questionnaire was used to the knowledge acquisition from experts [. With the paper questionnaire the basic knowledge related to the marine diesel exploitation was undertaken. The rule induction algorithms was used to knowledge acquisition from diagnostic database [. Training and test data were acquired from experiment on marine engine Sulzer 3AL 25/30. The paper proposes the original system of knowledge management for the marine diesel engine diagnosis. The system allows the collection of knowledge from different sources, its evaluation and update.

OPTIMAL ALGORITHM FOR PREDICTING STUDENTS ACADEMIC PERFORMANCE

Machine learning has been successfully applied to numerous domains such as pattern recognition, image recognition, fraud detection, medical diagnosis, banking, bioinformatics, commodity trading, computer games and various control applications. Recently, this paradigm is been employed to enhance and evaluate higher education tasks. The focus of this work is on identifying the optimal algorithm suitable for predicting first-year tertiary students academic performance based on their family background factors and previous academic achievement. One thousand five hundred (1,500) enrolment records of students admitted into computer science programme Babcock University, Nigeria between 2001 and 2010 was used. The students’ first year academic performance was measured by Cumulative Grade Point Average (CGPA) at the end of the first session and the previous academic achievement was measured by SSCE grade score and UME score. Waikato Environment for Knowledge Analysis (WEKA) was used to generate 10 classification models( five decision tree algorithms -Random forest, Random tree, J48, Decision stump and REPTree and five rule induction algorithms –JRip, OneR, ZeroR, PART, and Decision table)  and a multilayer perceptron, an artificial neural network function. These algorithms were compared using 10-fold cross validation and hold-out method considering accuracy level, confusion matrices and CPU time to determine the optimal model. This work will be taken further by designing a framework of predictive system based on the rules generated from the optimal model.

Simple Hybrid and Incremental Postpruning Techniques for Rule Induction

Pruning achieves the dual goal of reducing the complexity of the final hypothesis for improved comprehensibility, and improving its predictive accuracy by minimizing the overfitting due to noisy data. This paper presents a new hybrid pruning technique for rule induction, as well as an incremental postpruning technique based on a misclassification tolerance. Although both have been designed for RULES-7, the latter is also applicable to any rule induction algorithm in general. A thorough empirical evaluation reveals that the proposed techniques enable RULES-7 to outperform other state-of-the-art classification techniques. The improved classifier is also more accurate and up to two orders of magnitude faster than before.

Implementation and Evaluation of Rule Induction Algorithm with Association Rule Mining: A study in life insurance

Data Mining: extracting useful insights from large and detailed collections of data. With the increased possibilities in modern society for companies and institutions to gather data cheaply and efficiently, this subject has become of increasing importance. This interest has inspired a rapidly maturing research field with developments both on a theoretical, as well as on a practical level with the availability of a range of commercial tools. In this research work we use rule induction in data mining to obtain the accurate results with fast processing time. We using decision list induction algorithm to make order and unordered list of rules to coverage of maximum data from the data set. Using induction rule via association rule mining we can generate number of rules for training dataset to achieve accurate result with less error rate. We also use induction rule algorithms like confidence static and Shannon entropy to obtain the high rate of accurate results from the large dataset. This can also improves the traditional algorithms with good result.

Evaluation of Rule and Decision Tree Induction Algorithms for Generating Climate Change Scenarios for Temperature and Pan Evaporation on a Lake Basin

Climate change scenarios generated by general circulation models (GCMs) have too coarse a spatial resolution to be useful in planning disaster risk reduction and climate change adaptation strategies at regional to river/lake basin scales. This paper investigates the performances of existing state-of-the-art rule induction and tree algorithms, namely, single conjunctive rule learner, decision table, M5P model tree, decision stump, and REPTree. Downscaling models are developed to obtain projections of mean monthly maximum and minimum temperatures (Tmax and Tmin) as well as pan evaporation to lake-basin scale in an arid region in India using these algorithms. The predictor variables, such as air temperature, zonal wind, meridional wind, and geo-potential height, are extracted from the National Centers for Environmental Prediction (NCEP) reanalysis data set for the period 1948–2000 and from the simulations using third-generation Canadian coupled global climate models for emission scenarios for the period 2001–2100. A simple multiplicative shift was used for correcting predictand values. The performances of various models have been evaluated on several statistical performance parameters such as correlation coefficient, mean absolute error, and root mean square error. The M5P model tree algorithm was found to yield better performance among all other learning techniques explored in the present study. An increasing trend is observed for Tmax and Tmin for emission scenarios, whereas no trend has been observed for pan evaporation in the future.

CHIRA—Convex Hull Based Iterative Algorithm of Rules Aggregation

In the paper we present CHIRA, an algorithm performing decision rules aggregation. New elementary conditions, which are linear combinations of attributes may appear in rule premises during the aggregation, leading to so-called oblique rules. The algorithm merges rules iteratively, in pairs, according to a certain order specified in advance. It applies the procedure of determining convex hulls for regions in a feature space which are covered by aggregated rules. CHIRA can be treated as the generalization of rule shortening and joining algorithms which, unlike them, allows a rule representation language to be changed. Application of presented algorithm allows one to decrease a number of rules, especially in the case of data in which decision classes are separated by hyperplanes not perpendicular to the attribute axes. Efficiency of CHIRA has been verified on rules obtained by two known rule induction algorithms, RIPPER and q-ModLEM, run on 18 benchmark data sets. Additionally, the algorithm has been applied on synthetic data as well as on a real-life set concerning classification of natural hazards in hard-coal mines.