Symbolic Learning Algorithms Research Articles

Will Skynet Need a Librarian? A Literature Review of Machine LearninggBased Data Organization and Use Strategies.

The increasing use of machine learning in everyday life, from loan applications to morning coffee, has developed alongside the rocketing growth of data professionally, attempting to leave no stone unturned in the quest for new insights, and personally, as digitally-connected people both produce and consume more data than ever before. Learning how machine learning models and algorithms engage with data organization is important in understanding how to improve the utility and benefits of current and future machine learning uses. In surveying the literature landscape, we aim to look at the intersection of machine learning and data management from a few different fields for a comprehensive, albeit brief, view of this fascinating area of modern science and technology. In this literature review, databases, both relational and nonrelational, are covered, along with four types of machine learning: symbolic machine learning, neural networks, simulated evolution and genetic algorithms, and particle swarm optimization. We’ll also take a look at machine learning-focused information retrieval strategies and how selected fields utilize and advance machine learning.

Regression via Classification applied on software defect estimation

In this paper we apply Regression via Classification (RvC) to the problem of estimating the number of software defects. This approach apart from a certain number of faults, it also outputs an associated interval of values, within which this estimate lies with a certain confidence. RvC also allows the production of comprehensible models of software defects exploiting symbolic learning algorithms. To evaluate this approach we perform an extensive comparative experimental study of the effectiveness of several machine learning algorithms in two software data sets. RvC manages to get better regression error than the standard regression approaches on both datasets.

A machine learning method for extracting symbolic knowledge from recurrent neural networks.

Neural networks do not readily provide an explanation of the knowledge stored in their weights as part of their information processing. Until recently, neural networks were considered to be black boxes, with the knowledge stored in their weights not readily accessible. Since then, research has resulted in a number of algorithms for extracting knowledge in symbolic form from trained neural networks. This article addresses the extraction of knowledge in symbolic form from recurrent neural networks trained to behave like deterministic finite-state automata (DFAs). To date, methods used to extract knowledge from such networks have relied on the hypothesis that networks' states tend to cluster and that clusters of network states correspond to DFA states. The computational complexity of such a cluster analysis has led to heuristics that either limit the number of clusters that may form during training or limit the exploration of the space of hidden recurrent state neurons. These limitations, while necessary, may lead to decreased fidelity, in which the extracted knowledge may not model the true behavior of a trained network, perhaps not even for the training set. The method proposed here uses a polynomial time, symbolic learning algorithm to infer DFAs solely from the observation of a trained network's input-output behavior. Thus, this method has the potential to increase the fidelity of the extracted knowledge.

AN APPROACH TO EXPLAIN NEURAL NETWORKS USING SYMBOLIC ALGORITHMS

Although Artificial Neural Networks have been satisfactorily employed in several problems, such as clustering, pattern recognition, dynamic systems control and prediction, they still suffer from significant limitations. One of them is that the induced concept representation is not usually comprehensible to humans. Several techniques have been suggested to extract meaningful knowledge from trained networks. This paper proposes the use of symbolic learning algorithms, commonly used by the Machine Learning community, such as C4.5, C4.5rules and CN2, to extract symbolic representations from trained networks. The approach proposed is similar to that used by the Trepan algorithm, which extracts symbolic representations, expressed as decision trees, from trained networks. Experimental results are presented and discussed in order to compare the knowledge extracted from Artificial Neural Networks using the proposed approach and the Trepan approach. Results are compared regarding two aspects: fidelity and comprehensibility.

Using modular neural networks for business decisions

Understanding large amounts of information and efficiently using that information in improved decision making has become increasingly challenging as businesses collect terabytes of data. Businesses have turned to emerging technology including neural networks, symbolic learning, and genetic algorithms. In the current study, four classification methods were compared using results from an Indonesian contraceptive‐method preference survey. The four methods are linear discriminant analysis, quadratic discriminant analysis, backpropagation neural networks, and modular neural networks. The modular neural network is a more complex and less frequently used neural network model. This comparative study gives insight into its performance on classifying observations from a challenging data set, the 1987 National Indonesia Contraceptive Prevalence Survey.

Class-dependent discretization for inductive learning from continuous and mixed-mode data

Inductive learning systems can be effectively used to acquire classification knowledge from examples. Many existing symbolic learning algorithms can be applied in domains with continuous attributes when integrated with a discretization algorithm to transform the continuous attributes into ordered discrete ones. In this paper, a new information theoretic discretization method optimized for supervised learning is proposed and described. This approach seeks to maximize the mutual dependence as measured by the interdependence redundancy between the discrete intervals and the class labels, and can automatically determine the most preferred number of intervals for an inductive learning application. The method has been tested in a number of inductive learning examples to show that the class-dependent discretizer can significantly improve the classification performance of many existing learning algorithms in domains containing numeric attributes.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

STATLOG: COMPARISON OF CLASSIFICATION ALGORITHMS ON LARGE REAL-WORLD PROBLEMS

This paper describes work in the StatLog project comparing classification algorithms on large real-world problems. The algorithms compared were from symbolic learning (CART. C4.5, NewID, AC2,ITrule, Cal5, CN2), statistics (Naive Bayes, k-nearest neighbor, kernel density, linear discriminant, quadratic discriminant, logistic regression, projection pursuit, Bayesian networks), and neural networks (backpropagation, radial basis functions). Twelve datasets were used: five from image analysis, three from medicine, and two each from engineering and finance. We found that which algorithm performed best depended critically on the data set investigated. We therefore developed a set of data set descriptors to help decide which algorithms are suited to particular data sets. For example, data sets with extreme distributions (skew > l and kurtosis > 7) and with many binary/categorical attributes (>38%) tend to favor symbolic learning algorithms. We suggest how classification algorithms can be extended in a number of directions.

Machine learning for information retrieval: Neural networks, symbolic learning, and genetic algorithms

Information retrieval using probabilistic techniques has attracted significant attention on the part of researchers in information and computer science over the past few decades. In the 1980s, knowledge-based techniques also made an impressive contribution to “intelligent” information retrieval and indexing. More recently, information science researchers have turned to other newer artificial-intelligence-based inductive learning techniques including neural networks, symbolic learning, and genetic algorithms. These newer techniques, which are grounded on diverse paradigms, have provided great opportunities for researchers to enhance the information processing and retrieval capabilities of current information storage and retrieval systems. In this article, we first provide an overview of these newer techniques and their use in information science research. To familiarize readers with these techniques, we present three popular methods: the connectionist Hopfield network; the symbolic ID3/ID5R; and evolution-based genetic algorithms. We discuss their knowledge representations and algorithms in the context of information retrieval. Sample implementation and testing results from our own research are also provided for each technique. We believe these techniques are promising in their ability to analyze user queries, identify users' information needs, and suggest alternatives for search. With proper user-system interactions, these methods can greatly complement the prevailing full-text, keyword-based, probabilistic, and knowledge-based techniques. © 1995 John Wiley & Sons, Inc.

GANNET: A Machine Learning Approach to Document Retrieval

Information retrieval using probabilistic techniques has attracted significant attention on the part of researchers in information and computer science over the past few decades. In the 1980s, knowledge-based techniques also have made an impressive contribution to intelligent information retrieval and indexing. More recently, information science researchers have turned to other, newer artificial intelligence-based inductive learning techniques including neural networks, symbolic learning, and genetic algorithms. The newer techniques have provided great opportunities for researchers to experiment with diverse paradigms for effective information processing and retrieval.In this article we first provide an overview of newer techniques and their usage in information science research. We then present in detail the algorithms we adopted for a hybrid Genetic Algorithms and Neural Nets based system, called GANNET. GANNET performed concept (keyword) optimization for user-selected documents during information retrieval using the genetic algorithms. It then used the optimized concepts to perform concept exploration in a large network of related concepts through the Hopfield net parallel relaxation procedure. Based on a test collection of about 3,000 articles from DIALOG and an automatically created thesaurus, and using Jaccard's score as a performance measure, our experiment showed that GANNET improved the Jaccard's scores by about 50 percent and it helped identify the underlying concepts (keywords) that best describe the user-selected documents.

Inductive inference and neural nets

We present an interesting experiment comparing human learning (quick modelling) to symbolic learning (ID3), neural net learning (perceptron), and pattern learning (Pao-Hu). From this experiment we find that humans, neural nets, and the Pao-Hu procedure favour a decision tree different from that preferred by the ID3 machine learning algorithm. Implications for the ‘strong convergence hypothesis’ between neural network and machine learning are discussed, with emphasis here on a ‘stronger convergence hypothesis’ between neural networks and humans. We further compare the generalization properties of the four approaches, and we introduce a method of ‘backprediction’ as a measure of the relative quality of generalizations. The relationships of our findings to other recent investigations comparing symbolic and neural learning algorithms are discussed.

Symbolic and neural learning algorithms: an experimental comparison

Despite the fact that many symbolic and neural network (connectionist) learning algorithms address the same problem of learning from classified examples, very little is known regarding their comparative strengths and weaknesses. Experiments comparing the ID3 symbolic learning algorithm with the perception and backpropagation neural learning algorithms have been performed using five large, real-world data sets. Overall, backpropagation performs slightly better than the other two algorithms in terms of classification accuracy on new examples, but takes much longer to train. Experimental results suggest that backpropagation can work significantly better on data sets containing numerical data. Also analyzed empirically are the effects of (1) the amount of training data, (2) imperfect training examples, and (3) the encoding of the desired outputs. Backpropagation occasionally outperforms the other two systems when given relatively small amounts of training data. It is slightly more accurate than ID3 when examples are noisy or incompletely specified. Finally, backpropagation more effectively utilizes a “distributed” output encoding.