Conjunctive Clauses Research Articles

Satisfiability Threshold of Random Propositional S5 Theories

Modal logic S5, which isan important knowledge representation and reasoning paradigm, has been successfully applied in various artificial-intelligence-related domains. Similar to the random propositional theories in conjunctive clause form, the phase transition plays an important role in designing efficient algorithms for computing models of propositional S5 theories. In this paper, a new form of S5 formula is proposed, which fixes the number of modal operators and literals in the clauses of the formula. This form consists of reduced 3-3-S5 clauses of the form l1∨l2∨ξ, where ξ takes the form □(l3∨l4∨l5), ◊(l3∧l4∧l5), or a propositional literal, and li(1≤i≤5) is a classical literal. Moreover, it is demonstrated that any S5 formula can be translated into a set of reduced 3-3-S5 clauses while preserving its satisfiability. This work further investigates the probability of a random 3-c-S5 formula with c=1,2,3 being satisfied by random assignment. In particular, we show that the satisfiability threshold of random 3-1-S5 clauses is −ln2(1−Pd−Ps)ln78+Ps·ln34, where Ps and Pd denote the probabilities of different modal operators appearing in a clause. Preliminary experimental results on random 3-1-S5 formulas confirm this theoretical threshold.

The syntactic status of V-final conjunct clauses in Old English: the role of priming

This study is a corpus-based investigation of the use of the V-final (VF) order in Old English conjunct (or coordinate) clauses. The aim of the analysis is to determine which of the two hypotheses formulated in earlier studies of the subject finds more convincing data support in the available corpora of Old English. According to one interpretation, conjunct clauses are a subtype of main clauses, and the VF order is used in both groups to signal continuation in discourse, especially with punctual, dynamic and relatively heavy verbs. Under the other view, VF conjunct clauses are syntactically subordinate, with the coordinating conjunction blocking verb movement like a complementiser. The present study shows that while both hypotheses are descriptively adequate, the main mechanism responsible for the use of the VF order in conjunct clauses is syntactic priming, with the VF order activated by a trigger clause (usually subordinate) and spreading to the following conjunct clause(s), which often results in long chains of subsequent VF clauses.

A digital twin reproducing gene regulatory network dynamics of early Ciona embryos indicates robust buffers in the network.

How gene regulatory networks (GRNs) encode gene expression dynamics and how GRNs evolve are not well understood, although these problems have been studied extensively. We created a digital twin that accurately reproduces expression dynamics of 13 genes that initiate expression in 32-cell ascidian embryos. We first showed that gene expression patterns can be manipulated according to predictions by this digital model. Next, to simulate GRN rewiring, we changed regulatory functions that represented their regulatory mechanisms in the digital twin, and found that in 55 of 100 cases, removal of a single regulator from a conjunctive clause of Boolean functions did not theoretically alter qualitative expression patterns of these genes. In other words, we found that more than half the regulators gave theoretically redundant temporal or spatial information to target genes. We experimentally substantiated that the expression pattern of Nodal was maintained without one of these factors, Zfpm, by changing the upstream regulatory sequence of Nodal. Such robust buffers of regulatory mechanisms may provide a basis of enabling developmental system drift, or rewiring of GRNs without changing expression patterns of downstream genes, during evolution.

On the Convergence of Tsetlin Machines for the XOR Operator.

The Tsetlin Machine (TM) is a novel machine learning algorithm with several distinct properties, including transparent inference and learning using hardware-near building blocks. Although numerous papers explore the TM empirically, many of its properties have not yet been analyzed mathematically. In this article, we analyze the convergence of the TM when input is non-linearly related to output by the XOR-operator. Our analysis reveals that the TM, with just two conjunctive clauses, can converge almost surely to reproducing XOR, learning from training data over an infinite time horizon. Furthermore, the analysis shows how the hyper-parameter T guides clause construction so that the clauses capture the distinct sub-patterns in the data. Our analysis of convergence for XOR thus lays the foundation for analyzing other more complex logical expressions. These analyses altogether, from a mathematical perspective, provide new insights on why TMs have obtained the state-of-the-art performance on several pattern recognition problems.

Identifying neuroanatomical and behavioral features for autism spectrum disorder diagnosis in children using machine learning

Autism spectrum disorder (ASD) is a neurodevelopmental disorder that can cause significant social, communication, and behavioral challenges. Diagnosis of ASD is complicated and there is an urgent need to identify ASD-associated biomarkers and features to help automate diagnostics and develop predictive ASD models. The present study adopts a novel evolutionary algorithm, the conjunctive clause evolutionary algorithm (CCEA), to select features most significant for distinguishing individuals with and without ASD, and is able to accommodate datasets having a small number of samples with a large number of feature measurements. The dataset is unique and comprises both behavioral and neuroimaging measurements from a total of 28 children from 7 to 14 years old. Potential biomarker candidates identified include brain volume, area, cortical thickness, and mean curvature in specific regions around the cingulate cortex, frontal cortex, and temporal-parietal junction, as well as behavioral features associated with theory of mind. A separate machine learning classifier (i.e., k-nearest neighbors algorithm) was used to validate the CCEA feature selection and for ASD prediction. Study findings demonstrate how machine learning tools might help move the needle on improving diagnostic and predictive models of ASD.

Camera Work

Camera Work

Verb-final conjunct clauses in Old English prose

Abstract The aim of this study is to analyse intertextual differences in the use of V-final order in Old English conjunct clauses and to determine to what extent the source of these differences may be Latin influence. The analysis reveals that the frequency of V-final order in OE conjuncts is rather limited in most texts, and Bede’s Historia Ecclesiastica surfaces as the text in which the frequency of V-final conjunct clauses is exceptionally high. The study shows that the regular use of V-final order in Bede may be interpreted as a translation effect, with Latin inflating the frequency of the pattern in conjunct clauses, which means that the frequency of V-final conjunct clauses in early OE translations may not reflect native tendencies.

Using Tsetlin Machine to discover interpretable rules in natural language processing applications

AbstractTsetlin Machines (TM) use finite state machines for learning and propositional logic to represent patterns. The resulting pattern recognition approach captures information in the form of conjunctive clauses, thus facilitating human interpretation. In this work, we propose a TM‐based approach to three common natural language processing (NLP) tasks, namely, sentiment analysis, semantic relation categorization and identifying entities in multi‐turn dialogues. By performing frequent itemset mining on the TM‐produced patterns, we show that we can obtain a global and a local interpretation of the learning, one that mimics existing rule‐sets or lexicons. Further, we also establish that our TM based approach does not compromise on accuracy in the quest for interpretability, via comparison with some widely used machine learning techniques. Finally, we introduce the idea of a relational TM, which uses a logic‐based framework to further extend the interpretability.

Human-Level Interpretable Learning for Aspect-Based Sentiment Analysis

This paper proposes human-interpretable learning of aspect-based sentiment analysis (ABSA), employing the recently introduced Tsetlin Machines (TMs). We attain interpretability by converting the intricate position-dependent textual semantics into binary form, mapping all the features into bag-of-words (BOWs). The binary form BOWs are encoded so that the information on the aspect and context words are nearly lossless for sentiment classification. We further adapt the BOWs as input to the TM, enabling learning of aspect-based sentiment patterns in propositional logic. To evaluate interpretability and accuracy, we conducted experiments on two widely used ABSA datasets of SemEval 2014: Restaurant 14 and Laptop 14. The experiments show how each relevant feature takes part in conjunctive clauses that contain the context information for the corresponding aspect word, demonstrating human-level interpretability. At the same time, the obtained accuracy is competitive with existing neural network models, reaching 78.02% on Restaurant 14 and 73.51% on Laptop 14.

Clause Structure and Illocutionary Force in Medieval Gallo-Romance: Clitic Position in Old Occitan and Early Old French Sentential Coordination

Abstract This paper analyzes the position of object and adverbial clitic pronouns in coordinated affirmative verb-first declaratives introduced by e(t) “and” in Old Occitan and early Old French, a context in which clitics are variably preverbal or postverbal. An empirical study reveals that this variation is principled and reflects semantico-discursive properties in the same way in these two related and grammatically similar medieval Gallo-Romance varieties. On a theoretical level, I posit that preverbal clitics occur when conjunction occurs at the TP level, and postverbal clitics occur when conjunction occurs at the CP level, and that the choice of clause structure (TP vs. CP) for second conjunct clauses depends on illocutionary force, which in turn depends on discourse coherence relations and the semantics of verba dicendi (verbs of utterance).

Clause Structure and Illocutionary Force in Medieval Gallo-Romance: Clitic Position in Old Occitan and Early Old French Sentential Coordination

AbstractThis paper analyzes the position of object and adverbial clitic pronouns in coordinated affirmative verb-first declaratives introduced bye(t)“and” in Old Occitan and early Old French, a context in which clitics are variably preverbal or postverbal. An empirical study reveals that this variation is principled and reflects semantico-discursive properties in the same way in these two related and grammatically similar medieval Gallo-Romance varieties. On a theoretical level, I posit that preverbal clitics occur when conjunction occurs at the TP level, and postverbal clitics occur when conjunction occurs at the CP level, and that the choice of clause structure (TP vs. CP) for second conjunct clauses depends on illocutionary force, which in turn depends on discourse coherence relations and the semantics ofverba dicendi(verbs of utterance).

Extending the Tsetlin Machine With Integer-Weighted Clauses for Increased Interpretability

Building models that are both interpretable and accurate is an unresolved challenge for many pattern recognition problems. In general, rule-based and linear models lack accuracy, while deep learning interpretability is based on rough approximations of the underlying inference. However, recently, the rule-based Tsetlin Machines (TMs) have obtained competitive performance in terms of accuracy, memory footprint, and inference speed on diverse benchmarks (image classification, regression, natural language understanding, and game-playing). TMs construct rules using human-interpretable conjunctive clauses in propositional logic. These, in turn, are combined linearly to solve complex pattern recognition tasks. This paper addresses the accuracy-interpretability challenge in machine learning by introducing a TM with integer weighted clauses - the Integer Weighted TM (IWTM). The intent is to increase TM interpretability by reducing the number of clauses required for competitive performance. The IWTM achieves this by weighting the clauses so that a single clause can replace multiple duplicates. Since each TM clause is formed adaptively by a Tsetlin Automata (TA) team, identifying effective weights becomes a challenging online learning problem. We solve this problem by extending each team of TA with another kind of automaton: the stochastic searching on the line (SSL) automaton. We evaluate the performance of the new scheme empirically using five datasets, along with a study of interpretability. On average, IWTM uses 6.5 times fewer literals than the vanilla TM and 120 times fewer literals than a TM with real-valued weights. Furthermore, in terms of average memory usage and F1-Score, IWTM outperforms simple Multi-Layered Artificial Neural Networks, Decision Trees, Support Vector Machines, K-Nearest Neighbor, Random Forest, Gradient Boosted Trees (XGBoost), Explainable Boosting Machines (EBMs), as well as the standard and real-value weighted TMs. IWTM finally outperforms Neural Additive Models on Fraud Detection and StructureBoost on CA-58 in terms of Area Under Curve, while performing competitively on COMPAS.

A Tandem Evolutionary Algorithm for Identifying Causal Rules from Complex Data.

We propose a new evolutionary approach for discovering causal rules in complex classification problems from batch data. Key aspects include (a) the use of a hypergeometric probability mass function as a principled statistic for assessing fitness that quantifies the probability that the observed association between a given clause and target class is due to chance, taking into account the size of the dataset, the amount of missing data, and the distribution of outcome categories, (b) tandem age-layered evolutionary algorithms for evolving parsimonious archives of conjunctive clauses, and disjunctions of these conjunctions, each of which have probabilistically significant associations with outcome classes, and (c) separate archive bins for clauses of different orders, with dynamically adjusted order-specific thresholds. The method is validated on majority-on and multiplexer benchmark problems exhibiting various combinations of heterogeneity, epistasis, overlap, noise in class associations, missing data, extraneous features, and imbalanced classes. We also validate on a more realistic synthetic genome dataset with heterogeneity, epistasis, extraneous features, and noise. In all synthetic epistatic benchmarks, we consistently recover the true causal rule sets used to generate the data. Finally, we discuss an application to a complex real-world survey dataset designed to inform possible ecohealth interventions for Chagas disease.

Non-parametric learning of lifted Restricted Boltzmann Machines

We consider the problem of discriminatively learning Restricted Boltzmann Machines in the presence of relational data. Unlike previous approaches that employ a rule learner (for structure learning) and a weight learner (for parameter learning) sequentially, we develop a gradient-boosted approach that performs both simultaneously. Our approach learns a set of weak relational regression trees, whose paths from root to leaf are conjunctive clauses and represent the structure, and whose leaf values represent the parameters. When the learned relational regression trees are transformed into a lifted RBM, its hidden nodes are precisely the conjunctive clauses derived from the relational regression trees. This leads to a more interpretable and explainable model. Our empirical evaluations clearly demonstrate this aspect, while displaying no loss in effectiveness of the learned models.

Analysis and Recurrent Calculation of 8th Rank MBF of Maximal Types

This paper is a continuation of the study of monotone Boolean functions (MBFs) of maximal types using MBF partitioning into schemes. When factor out any of the variables out of the brackets, two MBFs are formed: left (in brackets) and right. It proved the possibility of such that take the one of the variables out of the brackets such that any conjunctive clause in the left MBF consists of fewer variables than any conjunctive clause in the right MBF. In addition, the left MBF absorbs the right MBF. For the first time, an important class of MBF of rank 8 - MBF of maximal types was studied and analyzed. The number of MBFs of maximal types of rank 8 and the number of isomorphic classes of such MBFs obtained from pairs of MBFs 7 rank are calculated. An example of the recursive construction MBF 8 rank is shown. Tables and schemes for MBF 8th rank are given. The dependences found between the maximal types MBF nth rank and rank n–1 make it possible to reduce the enumeration of MBFs by constructing rank 8 equivalence classes from rank 7 equivalence classes. The proposed methods are convenient for the analysis of large MBF ranks.

Using the Tsetlin Machine to Learn Human-Interpretable Rules for High-Accuracy Text Categorization With Medical Applications

Medical applications challenge today's text categorization techniques by demanding both high accuracy and ease-of-interpretation. Although deep learning has provided a leap forward in regard to accuracy, this leap comes at the sacrifice of interpretability. In this paper, we introduce a text categorization approach that leverages the recently introduced Tsetlin Machine to address this accuracy-interpretability challenge. Briefly, we represent the terms of a text as propositional variables. From these variables, we capture categories using simple propositional formulae, such as: IF “rash” AND “reaction” AND “penicillin” THEN Allergy. The Tsetlin Machine learns these formulae from labeled text, utilizing conjunctive clauses to represent the particular facets of each category. Therefore, also the absence of terms (negated features) can be used for categorization purposes. Our empirical comparisons with Naive Bayes classifiers, decision trees, linear support vector machines (SVMs), random forest, long short-term memory (LSTM) neural networks, and other techniques, are quite conclusive. Using relatively simple propositional formulae, the accuracy of the Tsetlin Machine either outperforms or performs approximately on par with the best evaluated methods on both the 20 Newsgroups and IMDb datasets, as well as on a clinical dataset containing authentic electronic health records (EHRs). On average, the Tsetlin Machine delivers the best recall and precision scores across the datasets. The main merit of the proposed approach is thus its capacity for producing human-interpretable rules, while at the same time achieving acceptable accuracy. We believe that our novel approach can have a significant impact on a wide range of text analysis applications, providing a promising starting point for deeper natural language understanding with the Tsetlin Machine.

The V-2 rule in Old English conjunct clauses

AbstractThis study shows that Old English conjunct clauses, i.e. main declarative clauses introduced by the coordinating conjunctionsandandac, resemble non-conjunct main clauses as far as the V-2 rule is concerned. Most importantly, this study reveals that the mechanism of SV inversion observed in OE conjunct clauses works according to all the principles defined for non-conjunct main clauses. The only difference, driven by the main discourse function of conjunct clauses, is that the clause-initial element in these clauses is usually the subject. However, if the subject is preceded by some other fronted constituent (e.g. an object, a complement, an adverb or a prepositional phrase), SV inversion is typical with nominal subjects, while personal pronoun subjects are only inverted if the clause-initial constituent belongs to a limited set of adverbs, i.e.þaandþonne(‘then’). In this way, this study reveals that the difference between Old English conjunct and non-conjunct main clauses is not as clear-cut as has traditionally been suggested.

An analysis of language errors in the middle school students' writings

This study analyzed the frequency of linguistic errors in middle school students` writing and correlation between frequency of errors and writing result. Previous studies have only introduced the types of errors in college students` writing. However, in order for linguistic error analysis to contribute to the improvement of writing quality, it is necessary to analyze what the errors are frequently. Additionally it will have a long-term educational effect when studied for school-age students. As a result of the analysis of 100 posts of middle school students, the linguistic errors of high frequency in middle school students were 1)inadequate conjunctive clause, 2)omission of essential sentence components, and 3)errors in honorific mood, and they all belong to the syntactic category. Especially, inappropriate sentences related to text and contextual categories need to think about the possibility that student writers have insufficient awareness of written language rules. The error of the syntactic category showed a significant correlation between the writing level and writing amount. The higher the writing level, the more the writing amount tended to increase. This means that the longer the length of the text, the greater the frequency of syntactic errors contained in it. However qualitative differences in linguistic errors between low and high-level students needs to be closely examined through correlations with the types of sentences or t-units generated in subsequent studies.

Old ‘truths’, new corpora: revisiting the word order of conjunct clauses in Old English

In Bech (2001a, 2001b), I took issue with the oft-repeated claim that Old English conjunct main clauses are commonly verb-final, and disproved it. However, the myth persists. In the meantime, theYork–Toronto–Helsinki Parsed Corpus of Old English Prose(YCOE, Tayloret al.2003) has been created, so the time has come to revisit this topic and consider it in light of new, extensive and generally accessible data. Using the YCOE corpus, I confirm and expand on Bech's (2001a, 2001b) empirical findings, showing that (i) OE conjunct clauses are neither typically verb-final nor verb-late, but they are more frequently verb-final and verb-late than non-conjunct clauses are; and (ii) verb-final and verb-late clauses are typically conjunct clauses. These two perspectives must be kept apart: in the first, the starting point is the entire body of conjunct clauses, and in the second it is the entire body of verb-final/verb-late clauses. I propose that the failure to distinguish between the two perspectives, i.e. whether it is conjunct clauses or word order that constitutes the point of departure, is the origin of the misconception concerning conjunct clauses and word order. In order to establish whether this distinction has been fuzzy all along, or whether it must be ascribed to distorted referencing in the course of a century of research, I trace the research on this topic back to the end of the nineteenth century. I show that the alleged verb-finality of conjunct clauses may be ascribed to awhisper-down-the-laneeffect – the retelling of the story has changed the story.

2-Xor Revisited: Satisfiability and Probabilities of Functions

The problem 2-Xor-Sat asks for the probability that a random expression, built as a conjunction of clauses $$x \oplus y$$xźy, is satisfiable. We revisit this classical problem by giving an alternative, explicit expression of this probability. We then consider a refinement of it, namely the probability that a random expression computes a specific Boolean function. The answers to both problems involve a description of 2-Xor expressions as multigraphs and use classical methods of analytic combinatorics by expressing probabilities through coefficients of generating functions.