Greater Expressive Power Research Articles

Rotation-equivariant graph neural networks for learning glassy liquids representations

The difficult problem of relating the static structure of glassy liquids and their dynamics is a good target for Machine Learning, an approach which excels at finding complex patterns hidden in data. Indeed, this approach is currently a hot topic in the glassy liquids community, where the state of the art consists in Graph Neural Networks (GNNs), which have great expressive power but are heavy models and lack interpretability. Inspired by recent advances in the field of Machine Learning group-equivariant representations, we build a GNN that learns a robust representation of the glass’ static structure by constraining it to preserve the roto-translation (SE(3)) equivariance. We show that this constraint significantly improves the predictive power at comparable or reduced number of parameters but most importantly, improves the ability to generalize to unseen temperatures. While remaining a Deep network, our model has improved interpretability compared to other GNNs, as the action of our basic convolution layer relates directly to well-known rotation-invariant expert features. Through transfer-learning experiments displaying unprecedented performance, we demonstrate that our network learns a robust representation, which allows us to push forward the idea of a learned structural order parameter for glasses.

Explainability analysis of neural network-based turbulence modeling for transonic axial compressor rotor flows

Model–consistent training has become trending for data-driven turbulence modeling since it can improve model generalizability and reduce data requirements by involving the Reynolds–averaged Navier–Stokes (RANS) equation during model learning. Neural networks are often used for the Reynolds stress representation due to their great expressive power, while they lack interpretability for the causal relationship between model inputs and outputs. Some post–hoc methods have been used to explain the neural network by indicating input feature importance. However, for the model–consistent training, the model explainability involves the analysis of both the neural network inputs and outputs. That is, the effects of model output on the RANS predictions should also be explained in addition to the input feature analysis. In this work, we investigate the explainability of the model–consistent learned model for the internal flow prediction of NASA Rotor 37 at its peak efficiency operating condition. The neural–network–based corrections for the Spalart–Allmaras turbulence model are learned from various experimental data based on the ensemble Kalman method. The learned model can noticeably improve the velocity prediction near the shroud. The explainability of the trained neural network is analyzed in terms of the model correction and the input feature importance. Specifically, the learned model correction increases the local turbulence production in the vortex breakdown region due to non–equilibrium effects, which capture the blockage effects near the shroud. Besides, the ratio of production to destruction and the helicity are shown to have relatively high importance for accurately predicting the compressor rotor flows based on the Shapley additive explanations method.

An Overview of Copy Move Forgery Detection Approaches

Images have greater expressive power than any other forms of documents. With the Internet, images are widespread in several applications. But the availability of efficient open-source online photo editing tools has made editing these images easy. The fake images look more appealing and original than the real image itself, which makes them indistinguishable and hence difficult to detect. The authenticity of digital images like medical reports, scan images, financial data, crime evidence, legal evidence, etc. is of high importance. Detecting the forgery of images is therefore a major research area. Image forgery is categorized as copy-move forgery, splicing, and retouching. In this work, a review of copy-move forgery is discussed along with the existing research on its detection and localization using both conventional and deep-learning mechanisms. The datasets used and challenges towards improving or developing novel algorithms are also presented.

Algebraic Perspectives on Signomial Optimization

Signomials are obtained by generalizing polynomials to allow for arbitrary real exponents. This generalization offers great expressive power but has historically sacrificed the organizing principle of “degree” that is central to polynomial optimization theory. We reclaim that principle here through the concept of signomial rings, which we use to derive complete convex relaxation hierarchies of upper and lower bounds for signomial optimization via sums of arithmetic-geometric exponentials (SAGE) nonnegativity certificates. The Positivstellensatz underlying the lower bounds relies on the concept of conditional SAGE and removes regularity conditions required by earlier works, such as convexity of the feasible set or Archimedeanity of its representing signomial inequalities. Through worked examples we illustrate the practicality of this hierarchy in areas such as chemical reaction network theory and chemical engineering. These examples include comparisons to direct global solvers (e.g., BARON and ANTIGONE) and the Lasserre hierarchy (where appropriate). The completeness of our hierarchy of upper bounds follows from a generic construction whereby a Positivstellensatz for signomial nonnegativity over a compact set provides for arbitrarily strong outer approximations of the corresponding cone of nonnegative signomials. While working toward that result, we prove basic facts on the existence and uniqueness of solutions to signomial moment problems.

On anti-occurrence of subsets of transitions in Petri net-based models of complex biological systems

Background and Objective:In the last two decades there can be observed a rapid development of systems biology. The basis of systems methods is a formal model of an analyzed system. It can be created in a language of some branch of mathematics and recently Petri net-based biological models seem to be especially promising since they have a great expressive power. One of the methods of analysis of such models is based on transition invariants. They correspond to some subprocesses which do not change a state of the modeled biological system. During such analysis, a need arose to study the subsets of transitions, what leads to interesting combinatorial problems — which have been considered in theory and practice. Methods & Results:Two problems of anti-occurrence were considered. These problems concern a set of transitions which is not a subset of any of t-invariant supports or is not a subset of t-invariant supports from some collection of such supports. They are defined in a formal way, their computational complexity is analyzed and an exact algorithm is provided for one of them. Conclusions:A comprehensive analysis of complex biological phenomena is challenging. Finding elementary processes that do not affect subprocesses belonging to the entire studied biological system may be necessary for a complete understanding of such a model and it is possible thanks to the proposed algorithm.

LSTM-Based Broad Learning System for Remaining Useful Life Prediction

Prognostics and health management (PHM) are gradually being applied to production management processes as industrial production is gradually undergoing a transformation, turning into intelligent production and leading to increased demands on the reliability of industrial equipment. Remaining useful life (RUL) prediction plays a pivotal role in this process. Accurate prediction results can effectively provide information about the condition of the equipment on which intelligent maintenance can be based, with many methods applied to this task. However, the current problems of inadequate feature extraction and poor correlation between prediction results and data still affect the prediction accuracy. To overcome these obstacles, we constructed a new fusion model that extracts data features based on a broad learning system (BLS) and embeds long short-term memory (LSTM) to process time-series information, named as the B-LSTM. First, the LSTM controls the transmission of information from the data to the gate mechanism, and the retained information generates the mapped features and forms the feature nodes. Then, the random feature nodes are supplemented by an activation function that generates enhancement nodes with greater expressive power, increasing the nonlinear factor in the network, and eventually the feature nodes and enhancement nodes are jointly connected to the output layer. The B-LSTM was experimentally used with the C-MAPSS dataset and the results of comparison with several mainstream methods showed that the new model achieved significant improvements.

A New Distance Measure to Rank Type-2 Intuitionistic Fuzzy Sets and Its Application to Multi-Criteria Group Decision Making

The type 2 intuitionistic fuzzy sets (T2IFS) have a greater expressive power in representing the uncertainty involved in the information given by the experts. A very few ranking methods have been proposed to compare the T2IFS based on distance measure. This paper proposes a new distance measure called Jaccard distance for type 2 intuitionistic fuzzy set. This method ranks the T2IFS based on the dissimilarity of the given T2IFS to the ideal set. The properties of distance measure have been proved. The efficiency of proposed method is validate by comparing with existing methods. It is observed that the proposed ranking can rank the T2IFS more effectively when compared to existing methods in some tested contexts. The proposed ranking method is applied in solving a multi criteria group decision making method and the results are analyzed. From the analysis it is observed that the ordering the alternatives using proposed method agrees with the human intuition.

Compilation of sparse array programming models

This paper shows how to compile sparse array programming languages. A sparse array programming language is an array programming language that supports element-wise application, reduction, and broadcasting of arbitrary functions over dense and sparse arrays with any fill value. Such a language has great expressive power and can express sparse and dense linear and tensor algebra, functions over images, exclusion and inclusion filters, and even graph algorithms. Our compiler strategy generalizes prior work in the literature on sparse tensor algebra compilation to support any function applied to sparse arrays, instead of only addition and multiplication. To achieve this, we generalize the notion of sparse iteration spaces beyond intersections and unions. These iteration spaces are automatically derived by considering how algebraic properties annotated onto functions interact with the fill values of the arrays. We then show how to compile these iteration spaces to efficient code. When compared with two widely-used Python sparse array packages, our evaluation shows that we generate built-in sparse array library features with a performance of 1.4× to 53.7× when measured against PyData/Sparse for user-defined functions and between 0.98× and 5.53× when measured against SciPy/Sparse for sparse array slicing. Our technique outperforms PyData/Sparse by 6.58× to 70.3×, and (where applicable) performs between 0.96× and 28.9× that of a dense NumPy implementation, on end-to-end sparse array applications. We also implement graph linear algebra kernels in our system with a performance of between 0.56× and 3.50× compared to that of the hand-optimized SuiteSparse:GraphBLAS library.

Non‐Differentiable Leaning of Quantum Circuit Born Machine with Genetic Algorithm

The Quantum Circuit Born Machine (QCBM) is a generative quantum machine learning model that can be efficiently trained and run on the NISQ-era quantum processors. It is likely that QCBM will be one of the first quantum machine learning models to find productive applications in quantitative finance as a powerful market generator. In this paper we test QCBM performance on a dataset of spot FX log-returns (heavy-tailed distribution) as well as specially constructed mixture of Normal distributions (which models spiky light-tailed distribution). The QCBM has greater expressive power than comparable classical neural networks such as restricted Boltzmann machine (RBM) and, therefore, has potential to demonstrate quantum advantage by generating high-quality samples from the learned empirical distribution of the market risk factors while using less computational resources than its classical counterpart. However, efficient training of QCBM remains a challenging problem. Traditional differentiable learning approach may not work well when the loss function is highly non-smooth. In such cases it may be more efficient to use the non-differentiable learning methods. This paper proposes a non-differentiable learning approach to the training of QCBM based on genetic glgorithm (GA). The paper also presents results of the numerical experiments which compare performance of QCBM trained with GA against performance of the equivalent classical RBM and investigates the question of GA convergence as a function of QCBM architecture and the choice of algorithm's hyperparameters.

Identity-aware Graph Neural Networks

Message passing Graph Neural Networks (GNNs) provide a powerful modeling framework for relational data. However, the expressive power of existing GNNs is upper-bounded by the 1-Weisfeiler-Lehman (1-WL) graph isomorphism test, which means GNNs that are not able to predict node clustering coefficients and shortest path distances, and cannot differentiate between different d-regular graphs. Here we develop a class of message passing GNNs, named Identity-aware Graph Neural Networks (ID-GNNs), with greater expressive power than the 1-WL test. ID-GNN offers a minimal but powerful solution to limitations of existing GNNs. ID-GNN extends existing GNN architectures by inductively considering nodes’ identities during message passing. To embed a given node, ID-GNN first extracts the ego network centered at the node, then conducts rounds of heterogeneous message passing, where different sets of parameters are applied to the center node than to other surrounding nodes in the ego network. We further propose a simplified but faster version of ID-GNN that injects node identity information as augmented node features. Alto- gether, both versions of ID-GNN represent general extensions of message passing GNNs, where experiments show that transforming existing GNNs to ID-GNNs yields on average 40% accuracy improvement on challenging node, edge, and graph property prediction tasks; 3% accuracy improvement on node and graph classification benchmarks; and 15% ROC AUC improvement on real-world link prediction tasks. Additionally, ID-GNNs demonstrate improved or comparable performance over other task-specific graph networks.

DNAQL: a query language for DNA sticker complexes

DNA computing has a rich history of computing paradigms with great expressive power. However, far less expressive power is needed for data manipulation. Indeed, the relational algebra, the yardstick of database systems, is expressible in first-order logic, and thus less powerful than Turing-complete models. Turing-complete DNA computing models have to account for many and varied scenarios. A DNA implementation of data manipulations might be nimbler and perform its operation faster than a Turing-complete DNA computing model. Hence, we propose a restrictive model for implementing data manipulation operations, focused on implementability in DNA. We call this model the sticker complex model. A forte of the sticker complex model, is its ability to detect when hybridization becomes an uncontrolled chain reaction. Such chain reactions make hybridization less predictable and thus less attractive for deterministic computations. Next, we define a query language on sticker complexes, called DNAQL. DNAQL is a typed, applicative functional programming language, powerful enough to simulate the relational algebra on sticker complexes. The type system enjoys a number of desirable properties such as soundness, maximality, and tightness.

Special Issue “Multi-Agent Systems”: Editorial

Multi-agent systems (MAS) are built around the central notions of agents, interaction, and environment. Agents are autonomous computational entities able to pro-actively pursue goals, and re-actively adapt to environment change. In doing so, they leverage on their social and situated capabilities: interacting with peers, and perceiving/acting on the environment. The relevance of MAS is steadily growing as they are extensively and increasingly used to model, simulate, and build heterogeneous systems across many different application scenarios and business domains, ranging from logistics to social sciences, from robotics to supply chain, and more. The reason behind such a widespread and diverse adoption lies in MAS great expressive power in modeling and actually supporting operational execution of a variety of systems demanding decentralized computations, reasoning skills, and adaptiveness to change, which are a perfect fit for MAS central notions introduced above. This special issue gathers 11 contributions sampling the many diverse advancements that are currently ongoing in the MAS field.

Non-Differentiable Learning of Quantum Circuit Born Machine with Genetic Algorithm

The Quantum Circuit Born Machine (QCBM) is a generative quantum machine learning model that can be efficiently trained and run on the NISQ era quantum processors. QCBM has greater expressive power than comparable classical neural networks such as Restricted Boltzmann Machine (RBM) and, therefore, has potential to demonstrate quantum advantage by generating high quality samples from the learned empirical distribution while using less computational resources than its classical counterpart. However, efficient training of QCBM remains a challenging problem. Traditional differentiable learning approach may not work well when the loss function is highly non-smooth. In such cases it may be more efficient to use the non-differentiable learning methods. This paper proposes a non-differentiable learning approach to the training of QCBM based on Genetic Algorithm (GA). The paper also presents results of the numerical experiments which compare performance of QCBM trained with GA against performance of the equivalent classical RBM and investigates the question of GA convergence as a function of QCBM architecture and the choice of algorithm’s hyperparameters.

A Robustness Analysis of Dynamic Boolean Models of Cellular Circuits.

With ever growing amounts of omics data, the next challenge in biological research is the interpretation of these data to gain mechanistic insights about cellular function. Dynamic models of cellular circuits that capture the activity levels of proteins and other molecules over time offer great expressive power by allowing the simulation of the effects of specific internal or external perturbations on the workings of the cell. However, the study of such models is at its infancy and no large-scale analysis of the robustness of real models to changing conditions has been conducted to date. Here we provide a computational framework to study the robustness of such models using a combination of stochastic simulations and integer linear programming techniques. We apply our framework to a large collection of cellular circuits and benchmark the results against randomized models. We find that the steady states of real circuits tend to be more robust in multiple aspects compared with their randomized counterparts.

Learning Adaptive Hidden Layers for Mobile Gesture Recognition

This paper addresses two obstacles hindering advances in accurate gesture recognition on mobile devices. First, gesture recognition performance is highly dependent on feature selection, but optimal features typically vary from gesture to gesture. Second, diverse user behaviors and mobile environments result in extremely large intra-class variations. We tackle these issues by introducing a new network layer, called an adaptive hidden layer (AHL), to generalize a hidden layer in deep neural networks and dynamically generate an activation map conditioned on the input. To this end, an AHL is composed of multiple neuron groups and an extra selector. The former compiles multi-modal features captured by mobile sensors, while the latter adaptively picks a plausible group for each input sample. The AHL is end-to-end trainable and can generalize an arbitrary subset of hidden layers. Through a series of AHLs, the great expressive power from exponentially many forward paths allows us to choose proper multi-modal features in a sample-specific fashion and resolve the problems caused by the unfavorable variations in mobile gesture recognition. The proposed approach is evaluated on a benchmark for gesture recognition and a newly collected dataset. Superior performance demonstrates its effectiveness.

Lewis meets Brouwer: Constructive strict implication

C.I. Lewis invented modern modal logic as a theory of “strict implication” ⥽. Over the classical propositional calculus one can as well work with the unary box connective. Intuitionistically, however, the strict implication has greater expressive power than □ and allows to make distinctions invisible in the ordinary syntax. In particular, the logic determined by the most popular semantics of intuitionistic K becomes a proper extension of the minimal normal logic of the binary connective. Even an extension of this minimal logic with the “strength” axiom, classically near-trivial, preserves the distinction between the binary and the unary setting. In fact, this distinction has been discovered by the functional programming community in their study of “arrows” as contrasted with “idioms”. Our particular focus is on arithmetical interpretations of intuitionistic ⥽ in terms of preservativity in extensions of HA, i.e., Heyting’s Arithmetic.

A Polynomial-time Regular Expressions Implementation

Regular expressions are a notation to define regular languagesin terms of simple composable operations. They are equivalentto finite automata in expressive power. In practice, however, modernregular expressions implementations diverge from the original theory.Most changes are made to allow greater expressive power. This conveniencecomes at the cost of making language membership a harderproblem than it could be. In many modern languages, the regex languagemembership is a NP-complete problem. Besides, the way they areimplemented sometimes causes expressions that could be processed inlinear time to take exponential time. This fact may be seen as a securityrisk for many applications that use regular expressions. In this work, wesuggest a simple implementation (based on Thompson’s ConstructionAlgorithm) that has superior worst-case performance than many popularimplementations. We also introduce a didactic notation for automatacreated by this algorithm that makes the adopted implementation easierto understand.

Traffic Behavior Recognition Using the Pachinko Allocation Model.

CCTV-based behavior recognition systems have gained considerable attention in recent years in the transportation surveillance domain for identifying unusual patterns, such as traffic jams, accidents, dangerous driving and other abnormal behaviors. In this paper, a novel approach for traffic behavior modeling is presented for video-based road surveillance. The proposed system combines the pachinko allocation model (PAM) and support vector machine (SVM) for a hierarchical representation and identification of traffic behavior. A background subtraction technique using Gaussian mixture models (GMMs) and an object tracking mechanism based on Kalman filters are utilized to firstly construct the object trajectories. Then, the sparse features comprising the locations and directions of the moving objects are modeled by PAM into traffic topics, namely activities and behaviors. As a key innovation, PAM captures not only the correlation among the activities, but also among the behaviors based on the arbitrary directed acyclic graph (DAG). The SVM classifier is then utilized on top to train and recognize the traffic activity and behavior. The proposed model shows more flexibility and greater expressive power than the commonly-used latent Dirichlet allocation (LDA) approach, leading to a higher recognition accuracy in the behavior classification.

The use of LPR (logic of plausible reasoning) to obtain information on innovative casting technologies

The paper presents a methodology for the application of a formalism of the logic of plausible reasoning (LPR) to create knowledge about a specific problem area. In this case, the methodology has been related to the task of obtaining information about the innovative casting technologies. In the search for documents, formulas created in terms of LPR have a much greater expressive power than the commonly used keywords. The discussion was illustrated with the results obtained using a pilot version of the original information tool.

Mining generalized temporal patterns based on fuzzy counting

Event-based sequences are a kind of pattern based on temporal associations with two essential characteristics: they are syntactically simple and have a great expressive power. For this reason, event-based sequence mining is an interesting solution to the problem of knowledge discovery in dynamic domains, mainly characterized by a time-varying nature. The inter-transactional model has led to the design of algorithms aimed to obtain this sort of patterns from time-stamped datasets. These algorithms extend the well-known Apriori algorithm, by explicitly adding the temporal context where associations among frequent events occurs. This leads to the possibility of extracting a larger number of patterns with a potential interest in decision making. However, its usefulness is diminished in those datasets where the characteristics of variability and uncertainty are present, which is a common issue in real domains. This is due to the rigidity of the counting method, which uses an exact measure of distance between temporal events. As a solution, we propose a generalization of the temporal mining process, which implies a relaxation of the counting method including the concept of approximate temporal distance between events. In particular, in this paper we present an algorithm, called TSETfuzzy-Miner, which incorporates a fuzzy-based counting technique in order to extract general, flexible, and practical temporal patterns taking into account the particular characteristics of real domains.