Expanded Search Space Research Articles

DREAM: Debugging and Repairing AutoML Pipelines

Deep Learning models have become an integrated component of modern software systems. In response to the challenge of model design, researchers proposed Automated Machine Learning (AutoML) systems, which automatically search for model architecture and hyperparameters for a given task. Like other software systems, existing AutoML systems have shortcomings in their design. We identify two common and severe shortcomings in AutoML, performance issue (i.e., searching for the desired model takes an unreasonably long time) and ineffective search issue (i.e., AutoML systems are not able to find an accurate enough model). After analyzing the workflow of AutoML, we observe that existing AutoML systems overlook potential opportunities in search space, search method, and search feedback, which results in performance and ineffective search issues. Based on our analysis, we design and implement DREAM, an automatic and general-purpose tool to alleviate and repair the shortcomings of AutoML pipelines and conduct effective model searches for diverse tasks. It monitors the process of AutoML to collect detailed feedback and automatically repairs shortcomings by expanding search space and leveraging a feedback-driven search strategy. Our evaluation results show that DREAM can be applied on two state-of-the-art AutoML pipelines and effectively and efficiently repair their shortcomings.

Optimizing Mean Fragment Size Prediction in Rock Blasting: A Synergistic Approach Combining Clustering, Hyperparameter Tuning, and Data Augmentation

Accurate estimation of the mean fragment size is crucial for optimizing open-pit mining operations. This study presents an approach that combines clustering, hyperparameter optimization, and data augmentation to enhance prediction accuracy using the Xtreme Gradient Boosting (XGBoost) regression model. A dataset of 110 blasts was divided into 97 blasts for training and testing, whereas a separate set of 13 new, unseen blasts was used to evaluate the robustness and generalization of the model. Hierarchical Agglomerative (HA) and K-means clustering algorithms were used, with HA clustering providing a higher cluster quality. To address class imbalance and improve model generalization, a synthetic minority oversampling technique for regression with Gaussian noise (SMOGN) was employed. Hyperparameter tuning was conducted using HyperOpt by comparing Random Search (RS) with the Advanced Tree-structured Parzen Estimator (ATPE). The combination of ATPE with HA clustering and SMOGN in an expanded search space produced the best results, achieving superior prediction accuracy and reliability. The proposed HAC1-SMOGN model, which integrates HA clustering, ATPE tuning, and SMOGN augmentation, achieved a mean squared error (MSE) of 0.0002 and an R2 of 0.98 on the test set. This study highlights the synergistic benefits of clustering, hyperparameter optimization, and data augmentation in enhancing machine learning models for regression tasks, particularly in scenarios with class imbalance or limited data.

Method for Expanding Search Space With Hybrid Operations in DynamicNAS

Method for Expanding Search Space With Hybrid Operations in DynamicNAS

Tree-shaped multiobjective evolutionary CNN for hyperspectral image classification

Convolutional neural networks (CNNs) have achieved significant performances in hyperspectral image (HSI) classification in recent years. However, designing a high-performance CNN depends on human expertise heavily, which usually takes considerable time and labor. With regard to reducing the burden of designing the networks, neural architecture search (NAS) has attracted increasing attention. A typical NAS approach aims to optimize the network architectures in a predefined search space with a suitable search algorithm automatically. However, the existing NAS work does not fully consider the spatial resolution and the spectral noise interference of HSIs. Furthermore, most NAS approaches use sequential blocks or cells to construct the networks, which are unsuitable for extracting multiscale features of HSIs and result in degraded performance. Considering the above challenges, we propose a tree-shaped multiobjective evolutionary CNN (TMOE-CNN) for HSI classification. An expanded search space is designed, which includes the image patch size and the channel number of the input image patches. A multibranch supernetwork structure is proposed, which resembles a tree as the fundamental architecture for the network block. The image patch size and the denoising strength of the input image patches can be established adaptively throughout the evolutionary search process. The tree-shaped networks can fuse multiscale features to enhance the capacity of the network for feature extraction. Additionally, we consider both the classification accuracy and the floating-point computational complexity in the environmental selection. It is helpful to find the networks with simple structure and low complexity while ensuring classification accuracy. Experiments on different HSI datasets show that TMOE-CNN can search CNNs with high accuracies and simple structures automatically.

Pump scheduling optimization in water distribution system based on mixed integer linear programming

The energy consumption in water distribution systems (WDSs) is significant. Improving the efficiency of pump operation can significantly reduce energy costs. However, optimal pump operation is a nonconvex mixed-integer nonlinear programming (MINLP) problem, which can be challenging to solve. A feasible approach is to linearize the problem and convert it into a mixed-integer linear programming (MILP) problem. However, this approach introduces many auxiliary variables, which can lead to inefficiency in finding the optimal solution due to the expanded search space. To address this issue, we propose a novel method for linearization of the original MINLP problem and a strategy that can adaptively adjust the number of piecewise linearization breakpoints. By reducing the number of auxiliary variables, our approach achieved competitive computing efficiency and the ability to save energy costs, as demonstrated in two benchmark instances. Furthermore, in a realistic large-scale WDS, our approach saved 9.83% more energy costs than the genetic algorithm and achieved a gap of only 7.36% from the lower bound.

An Improved Reptile Search Algorithm Based on Cauchy Mutation for Intrusion Detection

With the growth of the discipline of digital communication, the topic has acquired more attention in the cybersecurity medium. The Intrusion Detection (ID) system monitors network traffic to detect malicious activities. The paper introduces a novel Feature Selection (FS) approach for ID. Reptile Search Algorithm (RSA)—is a new optimization algorithm; in this method, each agent searches a new region according to the position of the host, which makes the algorithm suffers from getting stuck in local optima and a slow convergence rate. To overcome these problems, this study introduces an improved RSA approach by integrating Cauchy Mutation (CM) into the RSA’s structure. Thus, the CM can effectively expand search space and enhance the performance of the RSA. The developed RSA-CM is assessed on five publicly available ID datasets: KDD-CUP99, NSL-KDD, UNSW-NB15, CIC-IDS2017, and CIC-IDS2018 and two engineering problems. The RSA-CM is compared with the original RSA, and three other state-of-the-art FS methods, namely particle swarm optimization, grey wolf optimization, and multi-verse optimizer, and quantitatively is evaluated using fitness value, the number of selected optimum features, accuracy, precision, recall, and F1-score evaluation measures. The results reveal that the developed RSA-CM got better results than the other competitive methods applied for FS on the ID datasets and the examined engineering problems. Moreover, the Friedman test results confirm that RSA-CM has a significant superiority compared to other methods as an FS method for ID.

Development of an insilico model of eccrine sweat using molecular modelling techniques

Eccrine sweat is an ideal surrogate diagnostic biofluid for physiological and metabolic biomarkers for wearable biosensor design. Its periodic and non-invasive availability for candidate analytes such as glucose and cortisol along with limited correlation with blood plasma is of significant research interest. An insilico model of eccrine sweat can assist in the development of such wearable biosensors. In this regard, molecular modelling can be employed to observe the most fundamental interactions. Here, we determine a suitable molecular model for building eccrine sweat. The basic components of sweat are water and sodium chloride, in which glucose and other analytes are present in trace quantities. Given the wide range of water models available in the molecular dynamics space, in this study, we first validate the water models. We use three compounds to represent the base to build bulk sweat fluid and validate the force fields. We compare the self-diffusivity of water, glucose, sodium, and chloride ions as well as bulk viscosity values and present the results which are > 90% accurate as compared with the available literature. This validated insilico eccrine sweat model can serve as an aid to expedite the development de novo biosensors by addition of other analytes of interest e.g. cortisol, uric acid etc., simulate various temperatures and salt concentrations, expand search space for screening candidate target receptors by their binding affinity and assess the interference between competing species via simulations.

A tree search algorithm towards solving Ising formulated combinatorial optimization problems

Simulated annealing (SA) attracts more attention among classical heuristic algorithms because many combinatorial optimization problems can be easily recast as the ground state search problem of the Ising Hamiltonian. However, for practical implementation, the annealing process cannot be arbitrarily slow and hence, it may deviate from the expected stationary Boltzmann distribution and become trapped in a local energy minimum. To overcome this problem, this paper proposes a heuristic search algorithm by expanding search space from a Markov chain to a recursive depth limited tree based on SA, where the parent and child nodes represent the current and future spin states. At each iteration, the algorithm selects the best near-optimal solution within the feasible search space by exploring along the tree in the sense of “look ahead”. Furthermore, motivated by the coherent Ising machine (CIM), the discrete representation of spin states is relaxed to a continuous representation with a regularization term, which enables the use of the reduced dynamics of the oscillators to explore the surrounding neighborhood of the selected tree nodes. We tested our algorithm on a representative NP-hard problem (MAX-CUT) to illustrate the effectiveness of the proposed algorithm compared to semi-definite programming (SDP), SA, and simulated CIM. Our results show that with the primal heuristics SA and CIM, our high-level tree search strategy is able to provide solutions within fewer epochs for Ising formulated combinatorial optimization problems.

Joint Antenna and User Scheduling for MU MIMO Systems Using Efficient Binary Artificial Bee Colony Algorithm

In this paper, we present the MU (multi-user) MIMO (multiple-input & multiple-output) system which is capable to serve the maximum number of users without additional bandwidth or power budget. MIMO systems maximize the channel capability as per the DPC scheme (Dirty Paper Coding). According to DPC, several antennas distributed over the base station are used for serving multiple users simultaneously. Nevertheless, DPC is a comprehensive search algorithm employing sequences of encoded user data for transmitting information to multiple users. Due to increased user density and subsequent transmit antennas in the MU-MIMO system, the exhaustive search (i.e. DPC) becomes cumbersome with the expanded search space domain. This search task would not be completed within a coherence time frame of the packet data communication system. The required objective is to consider this as a cost function for optimization to maximize the realizable sum-rate (throughput) of the MU-MIMO systems. This paper illuminates the joint antenna and user scheduling (JAAU) process with binary artificial bee colony (binary ABC) accomplishing nearly 98–99% of system throughput with the suggestively decreased figure in computation complexity and run time. Binary ABC is observed to have the global optimum value swiftly within the stipulated packet time frame of next-generation wireless data communications. Moreover, binary ABC achieves an appreciable system capacity for MU-MIMO systems as compared with DPC (which attains the maximum system throughput with the highest complexity). These findings are corroborated with exhaustive simulation results presented in this article.

Maximizing Lifetime of Range-Adjustable Wireless Sensor Networks: A Neighborhood-Based Estimation of Distribution Algorithm.

Sensor activity scheduling is critical for prolonging the lifetime of wireless sensor networks (WSNs). However, most existing methods assume sensors to have one fixed sensing range. Prevalence of sensors with adjustable sensing ranges posts two new challenges to the topic: 1) expanded search space, due to the rise in the number of possible activation modes and 2) more complex energy allocation, as the sensors differ in the energy consumption rate when using different sensing ranges. These two challenges make it hard to directly solve the lifetime maximization problem of WSNs with range-adjustable sensors (LM-RASs). This article proposes a neighborhood-based estimation of distribution algorithm (NEDA) to address it in a recursive manner. In NEDA, each individual represents a coverage scheme in which the sensors are selectively activated to monitor all the targets. A linear programming (LP) model is built to assign activation time to the schemes in the population so that their sum, the network lifetime, can be maximized conditioned on the current population. Using the activation time derived from LP as individual fitness, the NEDA is driven to seek coverage schemes promising for prolonging the network lifetime. The network lifetime is thus optimized by repeating the steps of the coverage scheme evolution and LP model solving. To encourage the search for diverse coverage schemes, a neighborhood sampling strategy is introduced. Besides, a heuristic repair strategy is designed to fine-tune the existing schemes for further improving the search efficiency. Experimental results on WSNs of different scales show that NEDA outperforms state-of-the-art approaches. It is also expected that NEDA can serve as a potential framework for solving other flexible LP problems that share the same structure with LM-RAS.

The effectiveness of context-based change application on automatic program repair

An Automatic Program Repair (APR) technique is an implementation of a repair model to fix a given bug by modifying program behavior. Recently, repair models which collect source code and code changes from software history and use such collected resources for patch generation became more popular. Collected resources are used to expand the patch search space and to increase the probability that correct patches for bugs are included in the space. However, it is also revealed that navigation on such expanded patch search space is difficult due to the sparseness of correct patches in the space. In this study, we evaluate the effectiveness of Context-based Change Application (CCA) technique on change selection, fix location selection and change concretization, which are the key aspects of navigating patch search space. CCA collects abstract subtree changes and their AST contexts, and applies them to fix locations only if their contexts are matched. CCA repair model can address both search space expansion and navigation issues, by expanding search space with collected changes while narrowing down search areas in the search space based on contexts. Since CCA applies changes to a fix location only if their contexts are matched, it only needs to consider the same context changes for each fix location. Also, if there is no change with the same context as a fix location, this fix location can be ignored since it means that past patches did not modify such locations. In addition, CCA uses fine-grained changes preserving changed code structures, but normalizing user-defined names. Hence change concretization can be simply done by replacing normalized names with concrete names available in buggy code. We evaluated CCA’s effectiveness with over 54K unique collected changes (221K in total) from about 5K human-written patches. Results show that using contexts, CCA correctly found 90.1% of the changes required for test set patches, while fewer than 5% of the changes were found without contexts. We discovered that collecting more changes is only helpful if it is supported by contexts for effective search space navigation. In addition, CCA repair model found 44-70% of the actual fix locations of Defects4j patches more quickly compared to using SBFL techniques only. We also found that about 48% of the patches can be fully concretized using concrete names from buggy code.

Improved optimization of numerical association rule mining using hybrid particle swarm optimization and cauchy distribution

<div class="page" title="Page 1"><div class="layoutArea"><div class="column"><p>Particle Swarm Optimization (PSO) has been applied to solve optimization problems in various fields, such as Association Rule Mining (ARM) of numerical problems. However, PSO often becomes trapped in local optima. Consequently, the results do not represent the overall optimum solutions. To address this limitation, this study aims to combine PSO with the Cauchy distribution (PARCD), which is expected to increase the global optimal value of the expanded search space. Furthermore, this study uses multiple objective functions, i.e., support, confidence, comprehensibility, interestingness and amplitude. In addition, the proposed method was evaluated using benchmark datasets, such as the Quake, Basket ball, Body fat, Pollution, and Bolt datasets. Evaluation results were compared to the results obtained by previous studies. The results indicate that the overall values of the objective functions obtained using the proposed PARCD approach are satisfactory.</p></div></div></div>

A Novel Social Spider Optimization Algorithm for Large-Scale Economic Load Dispatch Problem

The paper develops an improved social spider optimization algorithm (ISSO) for finding optimal solutions of economic load dispatch (ELD) problems. Different ELD problem study cases can bring huge challenges for testing the robustness and effectiveness of the proposed ISSO method since discontinuous objective functions as well as complicated constraints are taken into account. The improved method is different from original social spider optimization algorithm (SSSO) by performing several modifications directly related to three processes of new solution generation. Namely, the proposed method keeps one formula for the first and the second generations and modify them effectively while SSSO has two different formulas for each generation. In the third generation, the proposed method applies a new formula for determining the mating radius of dominant males and females with the intent to expand search space and avoid falling into local zones. The modifications can support the proposed ISSO method find better solutions with faster manner than SSSO while the number of control parameters and the number of computational processes can be reduced. As a result, the proposed method can find much less generation cost and achieve faster search speeds than SSSO for all considered systems. On the other hand, the search ability evaluation of the proposed method is also given by comparing results with other existing methods available in previous studies. The proposed method can obtain approximate or better results and faster convergence than nearly all compared methods excluding for the last system. Consequently, the proposed ISSO method can be recommended to be a strong method for ELD problem and it can be tried for other mathematical problems in engineering.

Path Planning for Unmanned Underwater Vehicle Based on Improved Particle Swarm Optimization Method

Path planning of Unmanned Underwater Vehicle (UUV) is of considerable significance for the underwater navigation, the objective of the path planning is to find an optimal collision-free and the shortest trajectory from the start to the destination. In this paper, a new improved particle swarm optimization (IPSO) was proposed to process the global path planning in a static underwater environment for UUV. Firstly, the path planning principle for UUV was established, in which three cost functions, path length, exclusion potential field between the UUV and obstacle, and attraction potential field between UUV and destination, were considered and developed as an optimization objective. Then, on the basis of analysis traditional particle swarm optimization (PSO), the time-varying acceleration coefficients and slowly varying function were employed to improve performance of PSO, time-varying acceleration coefficients was utilized to balance the local optimum and global optimum, and slowly varying function was introduced into the updating formula of PSO to expand search space and maintain particle diversity. Finally, numerical simulations verify that, the proposed approach can fulfill path planning problems for UUN successfully.

Multilevel Cloud Detection in Remote Sensing Images Based on Deep Learning

Cloud detection is one of the important tasks for remote sensing image processing. In this paper, a novel multilevel cloud detection method based on deep learning is proposed for remote sensing images. First, the simple linear iterative clustering (SLIC) method is improved to segment the image into good quality superpixels. Then, a deep convolutional neural network (CNN) with two branches is designed to extract the multiscale features from each superpixel and predict the superpixel as one of three classes including thick cloud, thin cloud, and noncloud. Finally, the predictions of all the superpixels in the image yield the cloud detection result. In the proposed cloud detection framework, the improved SLIC method can obtain accurate cloud boundaries by optimizing initial cluster centers, designing dynamic distance measure, and expanding search space. Moreover, different from traditional cloud detection methods that cannot achieve multilevel detection of cloud, the designed deep CNN model can not only detect cloud but also distinguish thin cloud from thick cloud. Experimental results indicate that the proposed method can detect cloud with higher accuracy and robustness than compared methods.

Generating Fluttering Patterns with Low Autocorrelation for Coded Exposure Imaging

The performance of coded exposure imaging critically depends on finding good binary sequences. Previous coded exposure imaging methods have mostly relied on random searching to find the binary codes, but that approach can easily fail to find good long sequences, due to the exponentially expanding search space. In this paper, we present two algorithms for generating the binary sequences, which are especially well suited for generating short and long binary sequences, respectively. We show that the concept of low autocorrelation binary sequences, which has been successfully exploited in the field of information theory, can be applied to generate shutter fluttering patterns. We also propose a new measure for good binary sequences. Based on the new measure, we introduce two new algorithms for coded exposure imaging - a modified Legendre sequence method and a memetic algorithm. Experiments using both synthetic and real data show that our new algorithms consistently generate better binary sequences for the coded exposure problem, yielding better deblurring and resolution enhancement results compared to previous methods of generating the binary codes.

Adaptive mutation particle swarm algorithm with dynamic nonlinear changed inertia weight

A novel adaptive mutation particle swarm optimization algorithm with dynamic nonlinear changed inertia weight (AMPSO) was presented to accelerate searching speed and avoid premature convergence, which are mainly adjusted by inertia weight and information mutation. Firstly, the average particle spacing (APS) is employed to describe the population diversity, the smaller the APS, the more concentrated population and the worse species diversity. Then, inertia weight of each particle is updated as dynamic nonlinear based on APS to achieve a self-adaptive adjustment of global search ability and local search capabilities. Finally, information mutation based on slowly varying function (SV) introduced into the updating formula of PSO algorithm to expand search space. The proposed algorithm was tested compared with conventional PSO algorithms by three benchmark functions. The experiments show that the AMPSO has a stronger global optimization capability and higher search efficiency.

A formal concept analysis and semantic query expansion cooperation to refine health outcomes of interest.

BackgroundElectronic Health Records (EHRs) are frequently used by clinicians and researchers to search for, extract, and analyze groups of patients by defining Health Outcome of Interests (HOI). The definition of an HOI is generally considered a complex and time consuming task for health care professionals.MethodsIn our clinical note-based pharmacovigilance research, we often operate upon potentially hundreds of ontologies at once, expand query inputs, and we also increase the search space over clinical text as well as structured data. Such a method implies to specify an initial set of seed concepts, which are based on concept unique identifiers. This paper presents a novel method based on Formal Concept Analysis (FCA) and Semantic Query Expansion (SQE) to assist the end-user in defining their seed queries and in refining the expanded search space that it encompasses.ResultsWe evaluate our method over a gold-standard corpus from the 2008 i2b2 Obesity Challenge. This experimentation emphasizes positive results for sensitivity and specificity measures. Our new approach provides better recall with high precision of the obtained results. The most promising aspect of this approach consists in the discovery of positive results not present our Obesity NLP reference set.ConclusionsTogether with a Web graphical user interface, our FCA and SQE cooperation end up being an efficient approach for refining health outcome of interest using plain terms. We consider that this approach can be extended to support other domains such as cohort building tools.

Whole-genome analyses of whole-brain data: working within an expanded search space.

Large-scale comparisons of patients and healthy controls have unearthed genetic risk factors associated with a range of neurological and psychiatric illnesses. Meanwhile, brain imaging studies are increasing in size and scope, revealing disease and genetic effects on brain structure and function, and implicating neural pathways and causal mechanisms. With the advent of global neuroimaging consortia, imaging studies are now well powered to discover genetic variants that reliably affect the brain. Genetic analyses of brain measures from tens of thousands of people are being extended to test genetic associations with signals at millions of locations in the brain, and connectome-wide, genome-wide scans can jointly screen brain circuits and genomes; these analyses and others present new statistical challenges. There is a growing need for the community to establish and enforce standards in this developing field to ensure robust findings. Here we discuss how neuroimagers and geneticists have formed alliances to discover how genetic factors affect the brain. The field is rapidly advancing with ultra-high-resolution imaging and whole-genome sequencing. We recommend a rigorous approach to neuroimaging genomics that capitalizes on its recent successes and ensures the reliability of future discoveries.

NELDER-MEAD METHOD WITH LOCAL SELECTION USING NEIGHBORHOOD AND MEMORY FOR STOCHASTIC OPTIMIZATION

We consider the Nelder-Mead (NM) simplex algorithm for optimization of discrete-event stochastic simulation models. We propose new modifications of NM to reduce computational time and to improve quality of the estimated optimal solutions. Our means include utilizing past information of already seen solutions, expanding search space to their neighborhood and using adaptive sample sizes. We compare performance of these extensions on six test functions with 3 levels of random variations. We find that using past information leads to reduction of computational efforts by up to 20%. The adaptive modifications need more resources than the non-adaptive counterparts for up to 70% but give better-quality solutions. We recommend the adaptive algorithms with using memory with or without neighborhood structure.