Rule-based Learning Research Articles

Discovering rule-based learning systems for the purpose of music analysis

Music analysis and processing aim to understand the information retrieved from music (Music Information Retrieval). For the purpose of music data mining, machine learning (ML) methods or statistical approaches are often employed. Their primary task is to recognize the musical instrument sounds, the music genre, the emotion contained in music, the identification of audio, the assessment of audio content, and so on. In terms of computational approach, music databases contain imprecise, vague, and indiscernible data objects. Moreover, most of the machine learning algorithm outcomes are given as a black-box result. Also, underfitting or overfitting may occur, meaning that either the model description is not complex enough or the test set is too small or not sufficiently representative. Thus, the goal is to generalize the model. To overcome some of these problems, rule-based systems may be used, e.g., based on rough set theory that shows the outcome in the form of rules interconnecting the features retrieved from music. A potential of the rough set-based approach, a rule-based classifier, was shown in the context of music genre recognition. The results were analyzed in terms of the recognition rate and computation time efficiency.

Rule‐based classification of energy theft and anomalies in consumers load demand profile

The invent of advanced metering infrastructure (AMI) opens the door for a comprehensive analysis of consumers consumption patterns including energy theft studies, which were not possible beforehand. This study proposes a fraud detection methodology using data mining techniques such as hierarchical clustering and decision tree classification to identify abnormalities in consumer consumption patterns and further classify the abnormality type into the anomaly, fraud, high or low power consumption based on rule-based learning. The proposed algorithm uses real-time dataset of Nana Kajaliyala village, Gujarat, India. The focus has been on generalizing the algorithm for varied practical cases to make it adaptive towards non-malicious changes in consumer profile. Simultaneously, this study proposes a novel validation technique used for validation, which utilizes predicted profiles to ensure accurate bifurcation between anomaly and theft targets. The result exhibits high detection ratio and low false-positive ratio due to the application of appropriate validation block. The proposed methodology is also investigated from point of view of privacy preservation and is found to be relatively secure owing to low-sampling rates, minimal usage of metadata and communication layer. The proposed algorithm has an edge over state-of-the-art theft detection algorithms in detection accuracy and robustness towards outliers.

Construction of A Nautical Knowledge Graph Based on Multiple Data Sources

Nie, Q.; Sheng, X.; Zhang, N.; Wang, J.-J.; Shang, Y.-G., and Hu, F.-H., 2019. Construction of a nautical knowledge graph based on multiple data sources. In: Gong, D.; Zhu, H., and Liu, R. (eds.), Selected Topics in Coastal Research: Engineering, Industry, Economy, and Sustainable Development. Journal of Coastal Research, Special Issue No. 94, pp. 223–226. Coconut Creek (Florida), ISSN 0749-0208.To automatically recognize the nautical chart symbol is a difficult task; the first recognize step works well by the image recognition methods based on deep neural networks, however, these symbols usually have their own domain meanings which are different from their common meanings; a feasible method to improve the task is using domain knowledge to describe the domain meanings of these symbols. This paper proposes an automatic nautical knowledge graph construction method based on multiple data sources. The data sources adopted include structured data like data in relational databases, semi-structured data like data from web pages, and unstructured data like text. The author uses different methods to extract knowledge from these three types of data sources, including the rule-based method, wrapper method and machine learning method, etc. Then the author tests the proposed method by using it to establish a knowledge graph for the nautical industry. The established knowledge graph contains about 1,500 concepts and over 250 million hyponymy relations, about 80 million entities and about 30 million relations among the entities; and the average precision of these knowledge is above 0.9. The experimental result proves that the proposed method is precise and efficient.

Compact Belief Rule Base Learning for Classification with Evidential Clustering.

The belief rule-based classification system (BRBCS) is a promising technique for addressing different types of uncertainty in complex classification problems, by introducing the belief function theory into the classical fuzzy rule-based classification system. However, in the BRBCS, high numbers of instances and features generally induce a belief rule base (BRB) with large size, which degrades the interpretability of the classification model for big data sets. In this paper, a BRB learning method based on the evidential C-means clustering (ECM) algorithm is proposed to efficiently design a compact belief rule-based classification system (CBRBCS). First, a supervised version of the ECM algorithm is designed by means of weighted product-space clustering to partition the training set with the goals of obtaining both good inter-cluster separability and inner-cluster pureness. Then, a systematic method is developed to construct belief rules based on the obtained credal partitions. Finally, an evidential partition entropy-based optimization procedure is designed to get a compact BRB with a better trade-off between accuracy and interpretability. The key benefit of the proposed CBRBCS is that it can provide a more interpretable classification model on the premise of comparative accuracy. Experiments based on synthetic and real data sets have been conducted to evaluate the classification accuracy and interpretability of the proposal.

Interpretable genotype-to-phenotype classifiers with performance guarantees

Understanding the relationship between the genome of a cell and its phenotype is a central problem in precision medicine. Nonetheless, genotype-to-phenotype prediction comes with great challenges for machine learning algorithms that limit their use in this setting. The high dimensionality of the data tends to hinder generalization and challenges the scalability of most learning algorithms. Additionally, most algorithms produce models that are complex and difficult to interpret. We alleviate these limitations by proposing strong performance guarantees, based on sample compression theory, for rule-based learning algorithms that produce highly interpretable models. We show that these guarantees can be leveraged to accelerate learning and improve model interpretability. Our approach is validated through an application to the genomic prediction of antimicrobial resistance, an important public health concern. Highly accurate models were obtained for 12 species and 56 antibiotics, and their interpretation revealed known resistance mechanisms, as well as some potentially new ones. An open-source disk-based implementation that is both memory and computationally efficient is provided with this work. The implementation is turnkey, requires no prior knowledge of machine learning, and is complemented by comprehensive tutorials.

Extracting and reusing blocks of knowledge in learning classifier systems for text classification: a lifelong machine learning approach

Human beings follow a continuous learning paradigm, i.e., they learn to solve smaller and relatively easy problems, retain the learnt knowledge and apply that knowledge to learn and solve more complex and large-scale problems of the domain. Currently, most machine learning and evolutionary computing systems lack this ability to reuse the previous learnt knowledge. This paper presents a lifelong machine learning model for text classification that extracts the useful knowledge from simple problems of a domain and reuses the learnt knowledge to learn complex problems of the domain. The proposed approach adopts a rule-based learning classifier system, and a rich encoding scheme is used to extract and reuse building units of knowledge. The experimental results show that the continuous learning approach outperformed the baseline classifier system.

Novel representations that support rule-based categorization are acquired on-the-fly during category learning.

Humans learn categorization rules that are aligned with separable dimensions through a rule-based learning system, which makes learning faster and easier to generalize than categorization rules that require integration of information from different dimensions. Recent research suggests that learning to categorize objects along a completely novel dimension changes its perceptual representation, making it more separable and discriminable. Here, we asked whether such newly learned dimensions could support rule-based category learning. One group received extensive categorization training and a second group did not receive such training. Later, both groups were trained in a task that made use of the category-relevant dimension, and then tested in an analogical transfer task (Experiment 1) and a button-switch interference task (Experiment 2). We expected that only the group with extensive pre-training (with well-learned dimensional representations) would show evidence of rule-based behavior in these tasks. Surprisingly, both groups performed as expected from rule-based learning. A third experiment tested whether a single session (less than 1 h) of training in a categorization task would facilitate learning in a task requiring executive function. There was a substantial learning advantage for a group with brief pre-training with the relevant dimension. We hypothesize that extensive experience with separable dimensions is not required for rule-based category learning; rather, the rule-based system may learn representations "on the fly" that allow rule application. We discuss what kind of neurocomputational model might explain these data best.

Graph- and rule-based learning algorithms: a comprehensive review of their applications for cancer type classification and prognosis using genomic data.

Cancer is well recognized as a complex disease with dysregulated molecular networks or modules. Graph- and rule-based analytics have been applied extensively for cancer classification as well as prognosis using large genomic and other data over the past decade. This article provides a comprehensive review of various graph- and rule-based machine learning algorithms that have been applied to numerous genomics data to determine the cancer-specific gene modules, identify gene signature-based classifiers and carry out other related objectives of potential therapeutic value. This review focuses mainly on the methodological design and features of these algorithms to facilitate the application of these graph- and rule-based analytical approaches for cancer classification and prognosis. Based on the type of data integration, we divided all the algorithms into three categories: model-based integration, pre-processing integration and post-processing integration. Each category is further divided into four sub-categories (supervised, unsupervised, semi-supervised and survival-driven learning analyses) based on learning style. Therefore, a total of 11 categories of methods are summarized with their inputs, objectives and description, advantages and potential limitations. Next, we briefly demonstrate well-known and most recently developed algorithms for each sub-category along with salient information, such as data profiles, statistical or feature selection methods and outputs. Finally, we summarize the appropriate use and efficiency of all categories of graph- and rule mining-based learning methods when input data and specific objective are given. This review aims to help readers to select and use the appropriate algorithms for cancer classification and prognosis study.

Data Mining with Evolutionary Rule-Based Learning

Data Mining with Evolutionary Rule-Based Learning

Low-Dimensional Learning Control using Generic Signal Parametrizations

Iterative learning control (ILC) can yield superior performance for repetitive tasks while only requiring approximate models, making this control strategy very appealing for industry. However, applying it to non-linear systems involves solving of optimization problems, which limits the industrial uptake, especially for learning online to compensate for variations throughout the system’s lifetime. Industry tackles this by designing simple rule-based learning controllers. However, these are often designed in an ad-hoc manner, which potentially limits performance. In this paper, we will couple a low-dimensional parametrized learning control algorithm with a generic signal parametrization method on the basis of machine learning, and specifically using autoencoders. This will allow high control performance, while limiting implementational complexity and maintaining interpretability, paving the way for a higher industrial uptake of learning control for non-linear systems. We will illustrate the parametrized approach in simulation on a non-linear slider-crank system, and provide an example of using the learning approach to perform a tracking task for this system.

The Acquisition of Resultative Verb Compounds in Mandarin Chinese

Resultative verb compounds (RVCs) in Mandarin appear in a V1-V2 sequence and have three major subtypes—result-state, directional and completive RVCs with different degrees of compositionality depending on the extent to which the compound's meaning can be derived from the meanings of the component verbs. The derivation of the RVC has been analyzed as a lexical or syntactic process. Previous acquisition studies show that the RVC's compositionality and event structure are not fully acquired before the age of three. Findings conflict on whether the acquisition process revealed by early RVCs is usage-based or rule-based. We examined longitudinal data from two Mandarin-speaking children and administered a novel verb experiment on 32 children. The lexicalist approach to RVC formation is a better account for our data than the syntactic approach. Our data lend support to rule-based learning of RVCs. Children master the event structure of RVCs and its interaction with aspect gradually. The headedness of the RVC in relation to event structure depends on its type: V2 is more salient than V1 for the result-state RVC, while V1 is more salient than V2 for the directional RVC.

Characteristics of cognitive control in rule-based learning under pressure

Characteristics of cognitive control in rule-based learning under pressure

The effect of feedback delay on perceptual category learning and item memory: Further limits of multiple systems.

Delayed feedback during categorization training has been hypothesized to differentially affect 2 systems that underlie learning for rule-based (RB) or information-integration (II) structures. We tested an alternative possibility: that II learning requires more precise item representations than RB learning, and so is harmed more by a delay interval filled with a confusable mask. Experiments 1 and 2 examined the effect of feedback delay on memory for RB and II exemplars, both without and with concurrent categorization training. Without the training, II items were indeed more difficult to recognize than RB items, but there was no detectable effect of delay on item memory. In contrast, with concurrent categorization training, there were effects of both category structure and delayed feedback on item memory, which were related to corresponding changes in category learning. However, we did not observe the critical selective impact of delay on II classification performance that has been shown previously. Our own results were also confirmed in a follow-up study (Experiment 3) involving only categorization training. The selective influence of feedback delay on II learning appears to be contingent on the relative size of subgroups of high-performing participants, and in fact does not support that RB and II category learning are qualitatively different. We conclude that a key part of successfully solving perceptual categorization problems is developing more precise item representations, which can be impaired by delayed feedback during training. More important, the evidence for multiple systems of category learning is even weaker than previously proposed. (PsycINFO Database Record

Akaike Information Criterion-based conjunctive belief rule base learning for complex system modeling

Nonlinear complex system modeling has become the basis of many theoretical and practical problems, which requires balancing the correlations between the modeling accuracy and the modeling complexity. However, the two objectives may not be consistent with each other under many practical conditions, especially for complex systems with multiple influential factors. The belief rule base (BRB) has shown advantages in nonlinear complex system modeling under uncertainty. However, most of current works on BRB has focused only on the modeling accuracy. As such, an Akaike Information Criterion (AIC)-based objective, AICBRB, is deduced to represent both the modeling accuracy (denoted by the Mean Square Error (MSE)) and the modeling complexity (denoted by the number of the parameters). Based on the proposed AICBRB, a bi-level optimization model and a corresponding bi-level optimization algorithm are developed. Moreover, an empirical optimization path search strategy is proposed for upper-level optimization. The optimization path is comprised of multiple solutions with optimal performance. After the BRB learning process, both the structure and the parameters of BRB are optimized, which identifies the best decision structure of BRB. A numerical multi-extreme function case and a practical pipeline leak detection case are studied. The results show that an optimization path could be identified with a series of optimal solutions. With AICBRB as the objective, the jointly optimized BRB in the best decision structure can be obtained with an improved modeling accuracy as well as reduced modeling complexity.

Posthuman learning: AI from novice to expert?

Will robots ever be able to learn like humans? To answer that question, one first needs to ask: what is learning? Hubert and Stuart Dreyfus had a point when they claimed that computers and robots would never be able to learn like humans because human learning, after an initial phase of rule-based learning, is uncertain, context sensitive and intuitive (Dreyfus and Dreyfus in A five stage model of the mental activities involved in directed skill acquisition. (Supported by the U.S. Air Force, Office of Scientific Research (AFSC) under contract F49620-C-0063 with the University of California) Berkeley, February 1980. (Unpublished study). Washington, DC: Storming Media. https://www.stormingmedia.us/15/1554/A155480.html . Accessed 10 Oct 2017, 1980). I would add that learning also builds on prior learning, and that from the outset (birth), human learning is a socio-cultural materially grounded collective epistemology. This posthuman acknowledgement shifts the focus from the individual learner to learning within collective phenomena. Dreyfus and Dreyfus (1980) do not seem to emphasise the essentially social and cultural nature of the human condition. Learning theory (especially the Vygotskyan perspective), new materialism (especially as presented by the physicist Karen Barad) and postphenomenology (especially as presented by Don Ihde) have emphasised in different ways the materially based socio-cultural nature of human learning. They thereby point towards a ‘posthuman’ learning that is far from the machine-like or enhanced creature envisioned by singularists. Until robots are essentially social and ground their epistemologies in socio-cultural materiality, I suggest that human-like AI is not possible.

Automated detection of unusual soil moisture probe response patterns with association rule learning

In-situ field monitoring networks generate vast quantities of continuous data can help to improve the design, management, operation and maintenance of Green Infrastructure (GI) systems. However, such actions require efficient and reliable quality assurance quality control (QAQC). In this paper, we develop a rule-based learning algorithm involving Dynamic Time Warping (DTW) to investigate the feasibility of detecting anomalous responses from soil moisture probes using data collected from a GI site in Milwaukee, WI. As an enhancement to traditional QAQC methods which rely on individual time steps, this method converts the continuous time series into event sequences from which response patterns can be detected. Association rules are developed on both environmental features and event features. The results suggest that this method could be used to identify abnormal change patterns as compared to intra-site historical observations. Though developed for soil moisture, this method could easily be extended to apply on other continuous environmental datasets.

Comparison of rule induction, decision trees and formal concept analysis approaches for classification

Rule-based learning algorithms have higher transparency and easiness to interpret in comparison with neural networks and deep learning algorithms. These properties make it possible to effectively use such algorithms to solve descriptive tasks of data mining. The choice of an algorithm depends also on its ability to solve predictive tasks. The article compares the quality of the solution of the problems with binary and multiclass classification based on the experiments with six datasets from the UCI Machine Learning Repository. The authors investigate three algorithms: Ripper (rule induction), C4.5 (decision trees), In-Close (formal concept analysis). The results of the experiments show that In-Close demonstrates the best quality of classification in comparison with Ripper and C4.5, however the latter two generate more compact rule sets.

The Acquisition of Resultative Verb Compounds in Mandarin Chinese

Resultative verb compounds (RVCs) in Mandarin appear in a V1-V2 sequence and have three major subtypes—result-state, directional and completive RVCs with different degrees of compositionality depending on the extent to which the compound’s meaning can be derived from the meanings of the component verbs. The derivation of the RVC has been analyzed as a lexical or syntactic process. Previous acquisition studies show that the RVC’s compositionality and event structure are not fully acquired before the age of three. Findings conflict on whether the acquisition process revealed by early RVCs is usage-based or rule-based. We examined longitudinal data from two Mandarin-speaking children and administered a novel verb experiment on 32 children. The lexicalist approach to RVC formation is a better account for our data than the syntactic approach. Our data lend support to rule-based learning of RVCs. Children master the event structure of RVCs and its interaction with aspect gradually. The headedness of the RVC in relation to event structure depends on its type: V2 is more salient than V1 for the result-state RVC, while V1 is more salient than V2 for the directional RVC.

ConGEMs: Condensed Gene Co-Expression Module Discovery Through Rule-Based Clustering and Its Application to Carcinogenesis.

For transcriptomic analysis, there are numerous microarray-based genomic data, especially those generated for cancer research. The typical analysis measures the difference between a cancer sample-group and a matched control group for each transcript or gene. Association rule mining is used to discover interesting item sets through rule-based methodology. Thus, it has advantages to find causal effect relationships between the transcripts. In this work, we introduce two new rule-based similarity measures—weighted rank-based Jaccard and Cosine measures—and then propose a novel computational framework to detect condensed gene co-expression modules (s) through the association rule-based learning system and the weighted similarity scores. In practice, the list of evolved condensed markers that consists of both singular and complex markers in nature depends on the corresponding condensed gene sets in either antecedent or consequent of the rules of the resultant modules. In our evaluation, these markers could be supported by literature evidence, KEGG (Kyoto Encyclopedia of Genes and Genomes) pathway and Gene Ontology annotations. Specifically, we preliminarily identified differentially expressed genes using an empirical Bayes test. A recently developed algorithm—RANWAR—was then utilized to determine the association rules from these genes. Based on that, we computed the integrated similarity scores of these rule-based similarity measures between each rule-pair, and the resultant scores were used for clustering to identify the co-expressed rule-modules. We applied our method to a gene expression dataset for lung squamous cell carcinoma and a genome methylation dataset for uterine cervical carcinogenesis. Our proposed module discovery method produced better results than the traditional gene-module discovery measures. In summary, our proposed rule-based method is useful for exploring biomarker modules from transcriptomic data.

Korean 5W1H Extraction Using Rule-based and Machine Learning Methods

Korean 5W1H Extraction Using Rule-based and Machine Learning Methods