Structured Search Space Research Articles

Inverse catalysts: tuning the composition and structure of oxide clusters through the metal support

Computational modeling of metal–oxide interfaces is challenging due to the large search space of compositions and structures and the complexity of catalyst materials under operating conditions in general. In this work, we develop an efficient structure search workflow to discover chemically unique and relevant nanocluster geometries of inverse catalysts and apply it to ZnyOx and InyOx on Cu(111), Pd(111), and Au(111). We show that the workflow is successful in obtaining a large range of chemically distinct structures. Structural geometry trends are identified, including stable motifs such as tripod, rhombus, and pyramidal motifs. Using ab initio thermodynamics, we explore the in situ stability of the structures, including single-atom alloys, at a range of oxygen availabilities. This approach allows us to find trends such as the susceptibility to oxidation of the different systems and the range of stability of different cluster motifs. Our analysis highlights the importance of taking the diversity of sites exposed by metal–oxide interfaces into account in catalyst design studies.

Redefining Neural Architecture Search of Heterogeneous Multinetwork Models by Characterizing Variation Operators and Model Components.

With neural architecture search (NAS) methods gaining ground on manually designed deep neural networks-even more rapidly as model sophistication escalates-the research trend is shifting toward arranging different and often increasingly complex NAS spaces. In this conjuncture, delineating algorithms which can efficiently explore these search spaces can result in a significant improvement over currently used methods, which, in general, randomly select the structural variation operator, hoping for a performance gain. In this article, we investigate the effect of different variation operators in a complex domain, that of multinetwork heterogeneous neural models. These models have an extensive and complex search space of structures as they require multiple subnetworks within the general model in order to answer different output types. From that investigation, we extract a set of general guidelines whose application is not limited to that particular type of model and are useful to determine the direction in which an architecture optimization method could find the largest improvement. To deduce the set of guidelines, we characterize both the variation operators, according to their effect on the complexity and performance of the model; and the models, relying on diverse metrics which estimate the quality of the different parts composing it.

Autotuning Convolutions Is Easier Than You Think

A wide range of scientific and machine learning applications depend on highly optimized implementations of tensor computations. Exploiting the full capacity of a given processor architecture remains a challenging task, due to the complexity of the microarchitectural features that come into play when seeking near-peak performance. Among the state-of-the-art techniques for loop transformations for performance optimization, AutoScheduler [Zheng et al. 2020a ] tends to outperform other systems. It often yields higher performance as compared to vendor libraries, but takes a large number of runs to converge, while also involving a complex training environment. In this article, we define a structured configuration space that enables much faster convergence to high-performance code versions, using only random sampling of candidates. We focus on two-dimensional convolutions on CPUs. Compared to state-of-the-art libraries, our structured search space enables higher performance for typical tensor shapes encountered in convolution stages in deep learning pipelines. Compared to auto-tuning code generators like AutoScheduler, it prunes the search space while increasing the density of efficient implementations. We analyze the impact on convergence speed and performance distribution, on two Intel x86 processors and one ARM AArch64 processor. We match or outperform the performance of the state-of-the-art oneDNN library and TVM’s AutoScheduler, while reducing the autotuning effort by at least an order of magnitude.

FUBOCO: Structure Synthesis of Basic Op-Amps by FUnctional BlOck COmposition

This article presents a method to automatically synthesize the structure and initial sizing of an operational amplifier. It is positioned between approaches with fixed design plans and a small search space of structures and approaches with generic structural production rules and a large search space with technically impractical structures. The presented approach develops a hierarchical composition graph based on functional blocks that spans a search space of thousands of technically meaningful structure variants for single-output, fully differential, and complementary operational amplifiers. The search algorithm is a combined heuristic and enumerative process. The evaluation is based on circuit sizing with a library of behavioral equations of functional blocks. Formalizing the knowledge of functional blocks in op-amps for structural synthesis and sizing inherently reduces the search space and lessens the number of created topologies not fulfilling the specifications. Experimental results for the three op-amp classes are presented. An outlook how this method can be extended to multi-stage op-amps is given.

Latency-aware automatic CNN channel pruning with GPU runtime analysis

The huge storage and computation cost of convolutional neural networks (CNN) make them challenging to meet the real-time inference requirement in many applications. Existing channel pruning methods mainly focus on removing unimportant channels in a CNN model based on rule-of-thumb designs, using reduced floating-point operations (FLOPs) and parameter numbers to measure the pruning quality. The inference latency of pruned models is often overlooked. In this paper, we propose a latency-aware automatic CNN channel pruning method (LACP), which aims to search low latency and accurate pruned network structure automatically. We evaluate the inaccuracy of measuring pruning quality by FLOPs and the number of parameters, and use the model inference latency as the direct optimization metric. To bridge model pruning and inference acceleration, we analyze the inference latency of convolutional layers on GPU. Results show that the inference latency of convolutional layers exhibits a staircase pattern along with channel number due to the GPU tail effect. Based on that observation, we greatly shrink the search space of network structures. Then we apply an evolutionary procedure to search a computationally efficient pruned network structure, which reduces the inference latency and maintains the model accuracy. Experiments and comparisons with state-of-the-art methods on three image classification datasets show that our method can achieve better inference acceleration with less accuracy loss.

Leveraging Domain Knowledge in Multitask Bayesian Network Structure Learning

Network structure learning algorithms have aided network discovery in fields such as bioinformatics, neuroscience, ecology and social science. However, challenges remain in learning informative networks for related sets of tasks because the search space of Bayesian network structures is characterized by large basins of approximately equivalent solutions. Multitask algorithms select a set of networks that are near each other in the search space, rather than a score-equivalent set of networks chosen from independent regions of the space. This selection preference allows a domain expert to see only differences supported by the data. However, the usefulness of these algorithms for scientific datasets is limited because existing algorithms naively assume that all pairs of tasks are equally related. We introduce a framework that relaxes this assumption by incorporating domain knowledge about task-relatedness into the learning objective. Using our framework, we introduce the first multitask Bayesian network algorithm that leverages domain knowledge about the relatedness of tasks. We use our algorithm to explore the effect of task-relatedness on network discovery and show that our algorithm learns networks that are closer to ground truth than naive algorithms and that our algorithm discovers patterns that are interesting.

Improved K2 algorithm for Bayesian network structure learning

In this paper, we study the problem of learning the structure of Bayesian networks from data, which takes a dataset and outputs a directed acyclic graph. This problem is known to be NP-hard. Almost most of the existing algorithms for structure learning can be classified into three categories: constraint-based, score-based, and hybrid methods. The K2 algorithm, as a score-based algorithm, takes a random order of variables as input and its efficiency is strongly dependent on this ordering. Incorrect order of variables can lead to learning an incorrect structure. Therefore, the main challenge of this algorithm is strongly dependency of output quality on the initial order of variables. The main contribution of this paper is to derive a significant order of variables from the given dataset. Also, one of the significant challenges of structure learning is to find a practical structure learning approach to learn an optimal structure from complex and high-dimensional datasets in a reasonable time. We propose a new fast and straightforward algorithm for addressing this problem in a reasonable time. The proposed algorithm is based on an ordering by extracting strongly connected components of the graph built from data. We reduce the super-exponential search space of structures to the smaller space of nodes ordering. We evaluated the proposed algorithm using some standard benchmark datasets and compare the results with the results obtained from some state of the art algorithms. Finally, we show that the proposed algorithm is competitive with some algorithms for structure learning.

Mining relaxed functional dependencies from data

Relaxed functional dependencies (rfds) are properties expressing important relationships among data. Thanks to the introduction of approximations in data comparison and/or validity, they can capture constraints useful for several purposes, such as the identification of data inconsistencies or patterns of semantically related data. Nevertheless, rfds can provide benefits only if they can be automatically discovered from data. In this paper we present an rfd discovery algorithm relying on a lattice structured search space, previously used for fd discovery, new pruning strategies, and a new candidate rfd validation method. An experimental evaluation demonstrates the discovery performances of the proposed algorithm on real datasets, also providing a comparison with other algorithms.

Combinatorial search for selecting the structure of models of dynamical systems with equation discovery

Automated modeling aims at the induction of mathematical models, both their structure and parameter values, from time-series measurements of observed system variables. In this paper, we address the task of model structure selection, i.e., selecting an optimal structure from a user-specified finite set of alternative model structures, using various approaches to combinatorial search. We propose a mapping of the set of candidate model structures to a fixed-length, vector representation allowing the use of an arbitrary search algorithm as a solver of the structure selection task. We perform a comparative analysis of the performance of thirteen variants of several search algorithms, ranging from ones with high intensification, i.e., focus on neighborhood of the best candidate solutions, to ones with high diversification, i.e., focus on covering the entire search space. The empirical analysis involves eight tasks of reconstructing known models of dynamical systems from synthetic and measured data. The results of the analysis show that search algorithms involving moderate diversification methods have superior performance on the structure selection task. The empirical analysis also reveals that this finding is related to specific properties of the search space of candidate model structures.

Learning Parsimonious Classification Rules from Gene Expression Data Using Bayesian Networks with Local Structure.

The comprehensibility of good predictive models learned from high-dimensional gene expression data is attractive because it can lead to biomarker discovery. Several good classifiers provide comparable predictive performance but differ in their abilities to summarize the observed data. We extend a Bayesian Rule Learning (BRL-GSS) algorithm, previously shown to be a significantly better predictor than other classical approaches in this domain. It searches a space of Bayesian networks using a decision tree representation of its parameters with global constraints, and infers a set of IF-THEN rules. The number of parameters and therefore the number of rules are combinatorial to the number of predictor variables in the model. We relax these global constraints to a more generalizable local structure (BRL-LSS). BRL-LSS entails more parsimonious set of rules because it does not have to generate all combinatorial rules. The search space of local structures is much richer than the space of global structures. We design the BRL-LSS with the same worst-case time-complexity as BRL-GSS while exploring a richer and more complex model space. We measure predictive performance using Area Under the ROC curve (AUC) and Accuracy. We measure model parsimony performance by noting the average number of rules and variables needed to describe the observed data. We evaluate the predictive and parsimony performance of BRL-GSS, BRL-LSS and the state-of-the-art C4.5 decision tree algorithm, across 10-fold cross-validation using ten microarray gene-expression diagnostic datasets. In these experiments, we observe that BRL-LSS is similar to BRL-GSS in terms of predictive performance, while generating a much more parsimonious set of rules to explain the same observed data. BRL-LSS also needs fewer variables than C4.5 to explain the data with similar predictive performance. We also conduct a feasibility study to demonstrate the general applicability of our BRL methods on the newer RNA sequencing gene-expression data.

An ellipsoidal distance-based search strategy of ants for nonlinear single and multiple response optimization problems

Various continuous ant colony optimization (CACO) strategies are proposed by researchers to resolve continuous single response optimization problems. However, no such work is reported which also verifies suitability of CACO in case of both single and multiple response situations. In addition, as per literature survey, no variant of CACO can balance simultaneously all the three important aspects of an efficient search strategy, viz. escaping local optima, balancing between intensification and diversification scheme, and handling correlated variable search space structure. In this paper, a variant of CACO, so-called ‘CACO-MDS’ is proposed, which attempts to address all these three aspects. CACO-MDS strategy is based on a Mahalanobis distance-based diversification, and Nelder–Mead simplex-based intensification search scheme. Mahalanobis distance-based diversification search ensures exact measure of multivariate distance for correlated structured search space. The proposed CACO-MDS strategy is verified using fourteen single and multiple response multimodal function optimization test problems. A comparative analysis of CACO-MDS, with three different metaheuristic strategies, viz. ant colony optimization in real space (ACOR), a variant of local-best particle swarm optimization (SPSO) and simplex-simulated annealing (SIMPSA), also indicates its superiority in most of the test situations.

Bhageerath: an energy based web enabled computer software suite for limiting the search space of tertiary structures of small globular proteins

We describe here an energy based computer software suite for narrowing down the search space of tertiary structures of small globular proteins. The protocol comprises eight different computational modules that form an automated pipeline. It combines physics based potentials with biophysical filters to arrive at 10 plausible candidate structures starting from sequence and secondary structure information. The methodology has been validated here on 50 small globular proteins consisting of 2–3 helices and strands with known tertiary structures. For each of these proteins, a structure within 3–6 Å RMSD (root mean square deviation) of the native has been obtained in the 10 lowest energy structures. The protocol has been web enabled and is accessible at .

Semi-Automatic Composition of Loop Transformations for Deep Parallelism and Memory Hierarchies

Modern compilers are responsible for translating the idealistic operational semantics of the source program into a form that makes efficient use of a highly complex heterogeneous machine. Since optimization problems are associated with huge and unstructured search spaces, this combinational task is poorly achieved in general, resulting in weak scalability and disappointing sustained performance. We address this challenge by working on the program representation itself, using a semi-automatic optimization approach to demonstrate that current compilers offen suffer from unnecessary constraints and intricacies that can be avoided in a semantically richer transformation framework. Technically, the purpose of this paper is threefold: (1) to show that syntactic code representations close to the operational semantics lead to rigid phase ordering and cumbersome expression of architecture-aware loop transformations, (2) to illustrate how complex transformation sequences may be needed to achieve significant performance benefits, (3) to facilitate the automatic search for program transformation sequences, improving on classical polyhedral representations to better support operation research strategies in a simpler, structured search space. The proposed framework relies on a unified polyhedral representation of loops and statements, using normalization rules to allow flexible and expressive transformation sequencing. This representation allows to extend the scalability of polyhedral dependence analysis, and to delay the (automatic) legality checks until the end of a transformation sequence. Our work leverages on algorithmic advances in polyhedral code generation and has been implemented in a modern research compiler.

Prediction of substances of weak toxicity from chemical structures. an epidemiological approach

A global structure set of acute toxicity (LD50, mouse, iv) was constructed from databases. The common structural features of low toxicity substances were obtained from 1495 structure-toxicity data by applying the substructural balance method (SBM). This set contained 111 low toxicity ( > ca.1000 mg/kg) and 817 high toxicity substances (< ca.30 mg/kg). The result was expressed by a control chart which is made to control the structural causes to obtain the desired effect (low toxicity). Within the control limits, 100% of low toxic and 20% of high toxic substances were found. The control chart was tested with another set of 81 structures of low toxicity selected among 718 structures retrieved from RTECS. Seventy-eight structures were found within the control limits and the predictability was 96.3% (78/81). SBM employs a new paradigm of descriptors, the ratios of numbers of substructures, which were selected through a descriptor-screening process. The results confirm that the new paradigm is applicable for structure-based toxicity prediction. This “epidemiological” approach suggests key structural criteria where no reliable mechanism of toxicity is available and serves to reduce the search space of organic structures. A theory supporting SBM was discussed.

10.1162/153244304773936045

Two or more Bayesian network structures are Markov equivalent when the corresponding acyclic digraphs encode the same set of conditional independencies. Therefore, the search space of Bayesian network structures may be organized in equivalence classes, where each of them represents a different set of conditional independencies. The collection of sets of conditional independencies obeys a partial order, the so-called order. This paper discusses in depth the role that the inclusion order plays in learning the structure of Bayesian networks. In particular, this role involves the way a learning algorithm traverses the search space. We introduce a condition for traversal operators, the inclusion boundary condition, which, when it is satisfied, guarantees that the search strategy can avoid local maxima. This is proved under the assumptions that the data is sampled from a probability distribution which is faithful to an acyclic digraph, and the length of the sample is unbounded. The previous discussion leads to the design of a new traversal operator and two new learning algorithms in the context of heuristic search and the Markov Chain Monte Carlo method. We carry out a set of experiments with synthetic and real-world data that show empirically the benefit of striving for the inclusion order when learning Bayesian networks from data.

Navigation within a structured search space in layout problems

HeGeL-2 is a heuristic-based layout generator equipped with a nonserial dynamic programming algorithm. HeGeL-2 finds optimal solutions to plan-layout design (PLD) problems which are defined as problems that involve the allocation of a nontrivial number (ten or more) of design units in one application. The search strategy HeGeL-2 uses is similar in some respects and dissimilar in others to human search behavior, which has been simulated in an earlier version of HeGeL-2. These similarities include constraint relaxing strategies common to both versions. Dissimilarities concern claims of optimality and ‘goodness’ of solutions measured by an objective function based on soft constraints. In this paper, the general problem of navigation in a search space for layout problems is discussed in the context of a general-purpose formalism. Through this formalism several layout generators, including LOOS, WRIGHT, and HeGeL-2, are described.

Recognition of Own and Others' Utterances in a Natural Conversation

48 subjects discussed a criminal case in groups of 8. After two weeks, 31 were tested on recognition accuracy of utterances from the discussion. Recognition items were either the subjects' own statements, or statements from other participants in their group, or new. As expected, it was found that self-generated utterances were recognized as occurring more often during the conversation than other-generated utterances. A second factor varied focus of reference at recognition time: An item was attributed either to the subject himself or herself, or to some other unspecified person in the group. Primed self-reference at recognition time was expected to enhance recognition performance. This prediction was confirmed. Additionally, response confidence for self-referent items proved to be a more valid indicator of their correctness than confidence for other-referent items. The findings are discussed in terms of memory search. Focusing on the self as a highly structured search space enables the individual to recognize utterances-irrespective of their real source-more accurately than focusing on the memory representation of the group process as a whole.