Heuristic Search Process Research Articles

Memetic multilabel feature selection using pruned refinement process

With the growing complexity of data structures, which include high-dimensional and multilabel datasets, the significance of feature selection has become more emphasized. Multilabel feature selection endeavors to identify a subset of features that concurrently exhibit relevance across multiple labels. Owing to the impracticality of performing exhaustive searches to obtain the optimal feature subset, conventional approaches in multilabel feature selection often resort to a heuristic search process. In this context, memetic multilabel feature selection has received considerable attention because of its superior search capability; the fitness of the feature subset created by the stochastic search is further enhanced through a refinement process predicated on the employed multilabel feature filter. Thus, it is imperative to employ an effective refinement process that frequently succeeds in improving the target feature subset to maximize the benefits of hybridization. However, the refinement process in conventional memetic multilabel feature selection often overlooks potential biases in feature scores and compatibility issues between the multilabel feature filter and the subsequent learner. Consequently, conventional methods may not effectively identify the optimal feature subset in complex multilabel datasets. In this study, we propose a new memetic multilabel feature selection method that addresses these limitations by incorporating the pruning of features and labels into the refinement process. The effectiveness of the proposed method was demonstrated through experiments on 14 multilabel datasets.

Evolving filter criteria for randomly initialized network pruning in image classification

Network pruning aims to enhance the performance of deep neural networks by eliminating redundant components from the model. However, existing pruning methods typically require a well-trained model and employ fixed, single pruning criteria throughout the pruning cycles. To address these limitations, we propose a novel method called Evolutionary Filter Criteria (EvoFC). This method enables the automated search for the network pruning ratio and criterion during a population-based heuristic search process. We introduce a unique encoding space that represents the chosen pruning criterion and ratio for each layer, facilitating the acquisition of optimal architecture configurations for candidate networks during iterations. Additionally, we devise a novel weight inheritance mechanism to mitigate the computational burden associated with the population-based nature of the method, resulting in a significant reduction in overall training time. We validate our method by applying it to randomly initialized networks and conducting empirical experiments on CIFAR-10/100, ILSVRC-2012 and Places365 datasets. The results demonstrate that our method effectively reduces the number of FLOPs while striking a fine balance between accuracy and computational efficiency. This underscores the practical value of our method in optimizing performance while efficiently utilizing computational resources, particularly when pruning networks starting from random initialization.

Fusing entropy measures for dynamic feature selection in incomplete approximation spaces

The selection of informative and discriminative features from incomplete dynamic data has emerged as an essential problem because the data preparation in modern applications is a dynamic updating process, and the collected data are often fraught with missing or unobserved values. Using the effectiveness of entropy measures to quantify uncertainty information in incomplete approximation spaces under the framework of tolerance rough sets, a novel incremental feature selection approach is presented in this study from an information-theoretic perspective. Based on the updating patterns of tolerance classification and decision partition induced by conditional and decision features, a novel fused representation of entropy measures in an incomplete approximation space is proposed to accelerate the calculation of feature significance during the heuristic search process. A computationally efficient feature selection algorithm is developed by integrating the incremental fusing strategy of entropy when the data increases dynamically in size. Numerical experiments were performed on several benchmark datasets with data updating for different data sizes and incremental ratios to demonstrate the effectiveness of our method. The superiority of the proposed method is established extensively in terms of computational efficiency, cardinality and accuracy of the selected feature subset.

Faster Capacitated Arc Routing: A Sequence-to-Sequence Approach

The Capacitated Arc Routing Problem (CARP) is an NP-hard optimization problem that has been investigated for decades. Heuristic search methods are commonly used to solve it. However, given a CARP instance, most heuristic search algorithms require plenty of time to iteratively search for the solution from scratch, and hence may be impractical for emerging applications that need a solution to be obtained in a very short time period. In this work, a novel approach to efficiently solve CARP is presented. The proposed approach replaces the heuristic search process with the inference phase of a trained Deep Neural Network (DNN), which is trained to take a CARP instance as the input and outputs a solution to the instance. In this way, CARP could be solved by a direct mapping rather than by iterative search, and hence could be more efficient and more easily accelerated by the use of GPUs. Empirical study shows that the DNN-based solver can achieve significant speed-up with minor performance loss, and up to hundreds of times acceleration in extreme cases.

Modeling Information Pooling Bias in Incident Response Teams: An Agent Based Modeling Approach

Security analysts regularly correlate disparate incidents to detect cyber-attacks. However, past research shows that team-based incident correlation analysis may be affected by information pooling bias. This article presents findings from an agent-based model used to explore the cognitive processes hypothesized to be causing this bias during information exchange within a team. The model simulated information exchange between three analysts conducting incident correlation analysis by searching for information available with them about the different incidents. Three models of memory search process were compared: Random, Local, and Memory-aided search. Results from the simulation show that agents in a local search model, compared to memory-aided search model, shared more often the information known to majority in the team. Comparing model results with data from lab experiments suggest that teams, by default, may be employing a heuristic search process during information exchange leading to sub-optimal team processes and performance.

BLAST-QC: automated analysis of BLAST results

BackgroundThe Basic Local Alignment Search Tool (BLAST) from NCBI is the preferred utility for sequence alignment and identification for bioinformatics and genomics research. Among researchers using NCBI’s BLAST software, it is well known that analyzing the results of a large BLAST search can be tedious and time-consuming. Furthermore, with the recent discussions over the effects of parameters such as ‘-max_target_seqs’ on the BLAST heuristic search process, the use of these search options are questionable. This leaves using a stand-alone parser as one of the only options of condensing these large datasets, and with few available for download online, the task is left to the researcher to create a specialized piece of software anytime they need to analyze BLAST results. The need for a streamlined and fast script that solves these issues and can be easily implemented into a variety of bioinformatics and genomics workflows was the initial motivation for developing this software.ResultsIn this study, we demonstrate the effectiveness of BLAST-QC for analysis of BLAST results and its desirability over the other available options. Applying genetic sequence data from our bioinformatic workflows, we establish BLAST_QC’s superior runtime when compared to existing parsers developed with commonly used BioPerl and BioPython modules, as well as C and Java implementations of the BLAST_QC program. We discuss the ‘max_target_seqs’ parameter, the usage of and controversy around the use of the parameter, and offer a solution by demonstrating the ability of our software to provide the functionality this parameter was assumed to produce, as well as a variety of other parsing options. Executions of the script on example datasets are given, demonstrating the implemented functionality and providing test-cases of the program. BLAST-QC is designed to be integrated into existing software, and we establish its effectiveness as a module of workflows or other processes.ConclusionsBLAST-QC provides the community with a simple, lightweight and portable Python script that allows for easy quality control of BLAST results while avoiding the drawbacks of other options. This includes the uncertain results of applying the -max_target_seqs parameter or relying on the cumbersome dependencies of other options like BioPerl, Java, etc. which add complexity and run time when running large data sets of sequences. BLAST-QC is ideal for use in high-throughput workflows and pipelines common in bioinformatic and genomic research, and the script has been designed for portability and easy integration into whatever type of processes the user may be running.

Parallel heuristic search strategy based on a Bayesian approach for simultaneous recognition of contaminant sources and aquifer parameters at DNAPL-contaminated sites

In this study, we develop a parallel heuristic search strategy based on a Bayesian approach for simultaneously recognizing groundwater contaminant sources and aquifer parameters (unknown variables) at sites contaminated with dense non-aqueous phase liquids (DNAPLs). The parallel search strategy is time-consuming because thousands of simulation models must run in order to calculate the likelihood. Various stand-alone surrogate systems for the simulation models have been established, but they also have unavoidable limitations. Thus, we develop an optimal combined surrogate system by combining Gaussian process, kernel extreme learning machine, and support vector regression methods using a differential evolution algorithm with a variable mutation rate based on the rand-to-best/1/bin strategy, thereby improving the approximation accuracy of the surrogate system to the simulation model and significantly decreasing the high computational cost. Utilizing the optimal combined surrogate system reduced the CPU time by more than 400 times. In the iterative parallel heuristic search process, each round of iteration involves determining the candidate points and state transitions. The Monte Carlo approach is used widely for selecting candidate point, but this approach does not readily converge to the posterior distribution for unknown variables when the probability density function types are complex with weak search ergodicity. In order to improve the search ergodicity, we develop a particle swarm optimization algorithm with a non-linear decreasing inertia weight and Metropolis criterion, which is more suitable for unknown variables with complex probability density functions. The recognition results are obtained simultaneously when the iterative process terminates. We assess our proposed approaches based on a hypothetical case study at a three-dimensional site contaminated with DNAPLs. The results demonstrate that the parallel heuristic search strategy is helpful for the simultaneous recognition of DNAPL contaminant sources in groundwater and aquifer parameters.

Memory augmented hyper-heuristic framework to solve multi-disciplinary problems inspired by cognitive problem solving skills

This paper proposes a new framework, named Deja-Vu+, which is an extension of Deja Vu framework, a classic study on hyper-heuristic framework with 2R (Record and Recall) modules. Deja-Vu+ has the ability to handle two other domains, namely regression and unsupervised learning. The extension examines the strength of Deja-Vu+ for solving regression and unsupervised learning tasks. The regression problems are treated here as multiclass classification tasks, and unsupervised learning tasks are considered as clustering problems. The proposed framework is tested on a number of regression and unsupervised learning benchmark problems and has shown promising results to handle regression as classification. The framework attains an overall average accuracy of 70% for regression and clustering data sets. Deja-Vu+ is knowledge-rich hyper-heuristic framework, which is capable enough to transfer knowledge successfully. This knowledge transfer improves the performance of learning by avoiding the extensive heuristic search process. Our experimental results show that using previously attained knowledge to reduce the computational effort is beneficial in solving multi-disciplinary machine learning problems.

Non-recurrent congestion analysis using data-driven spatiotemporal approach for information construction

A systematic approach to quantify Incident-Induced Delay (IID) is proposed in this study. The paper complements existing literature by developing a data-driven method to dynamically determine the spatiotemporal extent of individual incidents. The information construction process can be further used to uncover a variety of features that are associated with any specific incidents for optimal freeway management. Additionally, this study contributes two particular highlights: secondary incident identification and K-Nearest Neighbor (KNN) pattern matching. Secondary incident identification, as a pre-processing for IID estimation, disentangles the convoluted influences of subsequent incidents. The proposed method uses KNN pattern matching, an essentially heuristic search process to separate the delay solely induced by incidents from the recurrent congestion. The proposed algorithm on IID quantification was implemented on Interstate 15 in the state of Utah using data obtained from 2013. Results and implications are presented. Hot spot analysis is conducted that can be potentially used for incident mitigation and to inform investment decisions. The proposed methodology is easily transferable to any traffic operation system that has access to sensor data at a corridor level.

The Central Role of Heuristic Search in Cognitive Computation Systems

This paper focuses on the relation of heuristic search and level of intelligence in cognitive computation systems. The paper begins with a review of the fundamental properties of a cognitive computation system, which is defined generally as a control system that generates goal-directed actions in response to environmental inputs and constraints. An important property of cognitive computations is the need to process local cues in symbol structures to access and integrate distal knowledge to generate a response. To deal with uncertainties involved in this local-to-distal processing, the system needs to perform heuristic search to locate and integrate the right set of distal structures. The level of intelligence of the system depends critically on the efficiency of the heuristic search process. It is argued that, for a bounded rationality system, the level of intelligence does not depend on how much search it needs to do to accomplish a task. Rather, the level of intelligence depends on how much search it does not need to do to achieve the same level of performance. Examples were discussed to illustrate this idea. The first two examples show how machines that play games like tic-tac-toe and chess rely heavily on the efficiency of the heuristic search algorithm to achieve better performance, demonstrating the relation of heuristic search and intelligence in a bounded rationality system. The second example shows how humans adapt to different information ecologies to perform information search on the Internet and how their performance improves over time, demonstrating how heuristic search can be improved in an adaptive rationality system. The two examples demonstrate how better search control knowledge and representations of task environment can improve the efficiency of heuristic search, thereby improving the intelligence of the system.

Content-Aware Photo Collage Using Circle Packing

In this paper, we present a novel approach for automatically creating the photo collage that assembles the interest regions of a given group of images naturally. Previous methods on photo collage are generally built upon a well-defined optimization framework, which computes all the geometric parameters and layer indices for input photos on the given canvas by optimizing a unified objective function. The complex nonlinear form of optimization function limits their scalability and efficiency. From the geometric point of view, we recast the generation of collage as a region partition problem such that each image is displayed in its corresponding region partitioned from the canvas. The core of this is an efficient power-diagram-based circle packing algorithm that arranges a series of circles assigned to input photos compactly in the given canvas. To favor important photos, the circles are associated with image importances determined by an image ranking process. A heuristic search process is developed to ensure that salient information of each photo is displayed in the polygonal area resulting from circle packing. With our new formulation, each factor influencing the state of a photo is optimized in an independent stage, and computation of the optimal states for neighboring photos are completely decoupled. This improves the scalability of collage results and ensures their diversity. We also devise a saliency-based image fusion scheme to generate seamless compositive collage. Our approach can generate the collages on nonrectangular canvases and supports interactive collage that allows the user to refine collage results according to his/her personal preferences. We conduct extensive experiments and show the superiority of our algorithm by comparing against previous methods.

Dynamics of Local Search Trajectory in Traveling Salesman Problem

This paper investigates dynamics of a local search trajectory generated by running the Or-opt heuristic on the traveling salesman problem. This study evaluates the dynamics of the local search heuristic by estimating the correlation dimension for the search trajectory, and finds that the local heuristic search process exhibits the transition from high-dimensional stochastic to low-dimensional chaotic behavior. The detection of dynamical complexity for a heuristic search process has both practical as well as theoretical relevance. The revealed dynamics may cast new light on design and analysis of heuristics and result in the potential for improved search process.

An algorithmic approach to concept exploration in a large knowledge network (automatic thesaurus consultation): Symbolic branch-and-bound search vs. connectionist Hopfield net activation

This paper presents a framework for knowledge discovery and concept exploration. In order to enhance the concept exploration capability of knowledge-based systems and to alleviate the limitations of the manual browsing approach, we have developed two spreading activation-based algorithms for concept exploration in large, heterogeneous networks of concepts (e.g., multiple thesauri). One algorithm, which is based on the symbolic AI paradigm, performs a conventional branch-and-bound search on a semantic net representation to identify other highly relevant concepts (a serial, optimal search process). The second algorithm, which is based on the neural network approach, executes the Hopfield net parallel relaxation and convergence process to identify “convergent” concepts for some initial queries (a parallel, heuristic search process). Both algorithms can be adopted for automatic, multiple-thesauri consultation. We tested these two algorithms on a large text-based knowledge network of about 13,000 nodes (terms) and 80,000 directed links in the area of computing technologies. This knowledge network was created from two external thesauri and one automatically generated thesaurus. We conducted experiments to compare the behaviors and performances of the two algorithms with the hypertext-like browsing process. Our experiment revealed that manual browsing achieved higher-term recall but lower-term precision in comparison to the algorithmic systems. However, it was also a much more laborious and cognitively demanding process. In document retrieval, there were no statistically significant differences in document recall and precision between the algorithms and the manual browsing process. In light of the effort required by the manual browsing process, our proposed algorithmic approach presents a viable option for efficiently traversing large-scale, multiple thesauri (knowledge network). © 1995 John Wiley & Sons, Inc.

Fuzzy computer-aided design of process manufacturing facilities

Preliminary design of process manufacturing facilities involves, among other things, the synthesis of detailed layout designs. In current practice, this spatial design process is very labor-intensive and expensive. This paper describes a prototype CAD system which models design decision-making, providing a computable framework for automation. The CAD system performs auto-elicitation of an expert's judgment in the form of fuzzy sets using interactive computer graphics. These fuzzy sets are then used in a heuristic search process employing multi-objective, non-linear optimization. Designs synthesized by this fuzzy CAD system are comparable to those generated “by hand,” and in some cases exceed a practitioner's design in quality. The CAD system, as presently constructed, provides multiple solutions. Conclusions and recommendations regarding processing speed and unrepresented heuristic content are made.

The heuristic search under conditions of error

By placing various restrictions on the heuristic estimator it is possible to constrain the heuristic search process to fit specific needs. This paper introduces a new restriction upon the heuristic, called the “bandwidth” condition, that enables the ordered search to better cope with time and space difficulties. In particular, the effect of error within the heuristic is considered in detail. Beyond this, the bandwidth condition quite naturally allows for the extension of the heuristic search to MIN/MAX trees. The resulting game playing algorithm affords many desirable practical features not found in minimax based techniques, as well as maintaining the theoretical framework of ordered searches. The development of this algorithm provides some additional insight to the general problem of searching game trees by showing that certain, somewhat surprising changes in the cost estimates are required to properly search the tree. Furthermore, the use of an ordered search of MIN/MAX trees brings about a rather provocative departure from the conventional approach to computer game playing.