Sub-symbolic Learning Research Articles

Conceptual orthospaces—Convexity meets negation

Neural networks and in general subsymbolic learning approaches perform well on usual learning tasks, but they are black boxes lacking desired properties such as explainability or trustworthiness. Fully integrated symbolic/subsymbolic systems account for these properties, e.g., through the injection of symbolic information into a subsymbolic learning framework. Systems relying on embeddings can be considered as specific examples of such systems where facts, expressed in some logic, are embedded into a continuous space. Instances mentioned in the facts are mapped to points and the relations/concepts mentioned in the facts are mapped to a region in that space. This allows to exploit geometric regularities in order to tackle typical learning tasks such as link prediction. Embeddings provide the first step towards filling the gap between qualitative, Tarskian style semantics, which is used for deductive reasoning over the facts, and quantitative structures, which are used for representing objects, relations, and concepts for learning purposes. However, to enable meaningful reasoning, embeddings of relations and concepts are not allowed to be shaped arbitrarily. Especially, convex sets turned out to be appropriate due to their computational advantages and due to their foundation in cognition. Convexity can be defined with a ternary betweenness relation and concepts can be defined as betweenness-closed (= convex) sets. Though many interesting phenomena of cognitive reasoning can be explained in such a framework, it is at least not obvious how to use betweenness for other, more logico-formal aspects of reasoning that, e.g., require defining logical operators. In particular, for the logical operator of negation other mathematical structures such as the orthoframes of Goldblatt (1974) [13] have proven more useful. In this paper, we provide results on the connection between convexity and orthoframes. In particular, we investigate the construction of convex concepts equipped with an orthogonality relation. We give a universal construction of a betweenness relation over an orthoframe and show that based on Euclidean betweenness (thus using the classical notion of convexity) and under some natural restrictions, convex cones are the only structures capable of modeling such a space.

Cognitive Modeling of Anticipation: Unsupervised Learning and Symbolic Modeling of Pilots' Mental Representations.

The ability to anticipate team members' actions enables joint action towards a common goal. Task knowledge and mental simulation allow for anticipating other agents' actions and for making inferences about their underlying mental representations. In human-AI teams, providing AI agents with anticipatory mechanisms can facilitate collaboration and successful execution of joint action. This paper presents a computational cognitive model demonstrating mental simulation of operators' mental models of a situation and anticipation of their behavior. The work proposes two successive steps: (1) A hierarchical cluster algorithm is applied to recognize patterns of behavior among pilots. These behavioral clusters are used to derive commonalities in situation models from empirical data (N = 13 pilots). (2) An ACT-R (adaptive control of thought - rational) cognitive model is implemented to mentally simulate different possible outcomes of action decisions and timing of a pilot. model tracing of ACT-R allows following up on operators' individual actions. Two models are implemented using the symbolic representations of ACT-R: one simulating normative behavior and the other by simulating individual differences and using subsymbolic learning. Model performance is analyzed by a comparison of both models. Results indicate the improved performance of the individual differences over the normative model and are discussed regarding implications for cognitive assistance capable of anticipating operatorbehavior.

Playing monotone games to understand learning behaviors

We deal with a special class of games against nature which correspond to subsymbolic learning problems where we know a local descent direction in the error landscape but not the amount gained at each step of the learning procedure. Namely, Alice and Bob play a game where the probability of victory grows monotonically by unknown amounts with the resources each employs. For a fixed effort on Alice’s part Bob increases his resources on the basis of the results of the individual contests (victory, tie or defeat). Quite unlike the usual ones in game theory, his aim is to stop as soon as the defeat probability goes under a given threshold with high confidence. We adopt such a game policy as an archetypal remedy to the general overtraining threat of learning algorithms. Namely, we deal with the original game in a computational learning framework analogous to the Probably Approximately Correct formulation. Therein, a wise use of a special inferential mechanism (known as twisting argument) highlights relevant statistics for managing different trade-offs between observability and controllability of the defeat probability. With similar statistics we discuss an analogous trade-off at the basis of the stopping criterion of subsymbolic learning procedures. As a conclusion, we propose a principled stopping rule based solely on the behavior of the training session, hence without distracting examples into a test set.

Ga s rule for knowledge discovery

This article presents a new approach for structural rule extraction and knowledge discovery by means of Structural Galois Lattice and genetic algorithms. The approach synthesizes symbolic learning in feature extraction as a pre-processing and a subsymbolic learning as a post-processing for extracting rules. Structural Galois Lattice was used to represent structural patterns, perform classification tasks, and extract features. These structural patterns were described by labeled graphs. The proposed method, GAsRule, is based on genetic algorithms which were adapted to 1) allow pattern recognition--this is done by matching the rule antecedents with the rule precedent (i.e., paths/graphs); 2) preserving the syntax and semantics of the context of description; and 3) evaluate rule sets for knowledge discovery and evolve new rules sets for prediction. The goal of our experiment is to solve the fundamental issue of extracting structural rules in structural pattern recognition. Experiments were based upon the examples of arch definition, which are widely used in machine learning. We hope to extend this method to real-world data sets in future work.

Gaining degrees of freedom in subsymbolic learning

We provide some theoretical results on sample complexity of PAC learning when the hypotheses are given by subsymbolical devices such as neural networks. In this framework we give new foundations to the notion of degrees of freedom of a statistic and relate it to the complexity of a concept class. Thus, for a given concept class and a given sample size, we discuss the efficiency of subsymbolical learning algorithms in terms of degrees of freedom of the computed statistic. In this setting we appraise the sample complexity overhead coming from relying on approximate hypotheses and display an increase in the degrees of freedom yield by embedding available formal knowledge into the algorithm. For known sample distribution, these quantities are related to the learning approximation goal and a special production prize is shown. Finally, we prove that testing the approximation capability of a neural network generally demands smaller sample size than training it.

Symbolic pattern manipulation of Karnaugh-Veitch maps for pneumatic circuits

Pneumatics continue to play a vital role in low-cost automation. The designing of pneumatic control circuits has to date been a slow manual process. This paper describes the computational symbolic manipulation of the Karnaugh-Veitch (KV) map which is the heart of the prototype expert system called PNEUMAES. The symbolic manipulation of a KV map is governed by two sets of generic rules for signal flow plotting and for logic equation minimisation applicable for complex pneumatic circuits. As the complexity of the circuit increases, the symbolic manipulation of a KV map leads to the combinatorial explosion problem. Because of this problem, PNEUMAES can only automatically generate pure pneumatic circuit design equations which will yield minimised circuit configuration for up to four cylinders with auxiliary control valves. A case study is included and issues and problems relating to the implementation of the KV map are discussed. Symbolic and sub-symbolic learning approaches are suggested as a means by which the search space of the symbolic patterns of the KV map can be pruned.

Machine learning of material behaviour knowledge from empirical data

Symbolic machine learning techniques can extract flexible and comprehensible knowledge from empirical data of material behaviour. The diversity of symbolic machine learning techniques offers potential to match the requirements of many tasks when models of material behaviour need to be created from data. We develop a series of steps for generating material behaviour knowledge from empirical data and exemplify some of them on several small datasets. We discuss some of the issues that govern knowledge extraction and, as a by-product, demonstrate that symbolic learning techniques are functionally superior to sub-symbolic learning for the task of comprehensible knowledge extraction.