High-dimensional Search Space Research Articles

A comprehensive learning based swarm optimization approach for feature selection in gene expression data

Gene expression data analysis is challenging due to the high dimensionality and complexity of the data. Feature selection, which identifies relevant genes, is a common preprocessing step. We propose a Comprehensive Learning-Based Swarm Optimization (CLBSO) approach for feature selection in gene expression data. CLBSO leverages the strengths of ants and grasshoppers to efficiently explore the high-dimensional search space. Ants perform local search and leave pheromone trails to guide the swarm, while grasshoppers use their ability to jump long distances to explore new regions and avoid local optima. The proposed approach was evaluated on several publicly available gene expression datasets and compared with state-of-the-art feature selection methods. CLBSO achieved an average accuracy improvement of 15% over the original high-dimensional data and outperformed other feature selection methods by up to 10%. For instance, in the Pancreatic cancer dataset, CLBSO achieved 97.2% accuracy, significantly higher than XGBoost-MOGA's 84.0%. Convergence analysis showed CLBSO required fewer iterations to reach optimal solutions. Statistical analysis confirmed significant performance improvements, and stability analysis demonstrated consistent gene subset selection across different runs. These findings highlight the robustness and efficacy of CLBSO in handling complex gene expression datasets, making it a valuable tool for enhancing classification tasks in bioinformatics.

Ultrafast artificial intelligence: Machine learning with atomic-scale quantum systems

Abstract We train a model atom to recognize hand-written digits in the range 0-9, employing intense light--matter interaction as a computational resource. For training, the images of the digits are converted into shaped laser pulses (data input pulses). Simultaneously with an input pulse, another shaped pulse (program pulse), polarized in the orthogonal direction, is applied to the atom and the system evolves quantum mechanically according to the time-dependent Schr"odinger equation. The purpose of the optimal program pulse is to direct the system into specific atomic final states (classification states) that correspond to the input digits. 
A success rate of about 40\% is achieved when using a basic optimization scheme that might be limited by the computational resources for finding the optimal program pulse in a high-dimensional search space. Our key result is the capability of the laser-programmed atom to generalize, i.e.,~the classification of unseen images is improved by training. This atom-sized machine-learning image-recognition scheme operates on time scales down to tens of femtoseconds, is scalable towards larger (e.g. molecular) systems, and is readily reprogrammable towards other learning/classification tasks.

Parallel hyperparameter optimization of spiking neural networks

Hyperparameter optimization of spiking neural networks (SNNs) is a difficult task which has not yet been deeply investigated in the literature. In this work, we designed a scalable constrained Bayesian based optimization algorithm that prevents sampling in non-spiking areas of an efficient high dimensional search space. These search spaces contain infeasible solutions that output no or only a few spikes during the training or testing phases, we call such a mode a “silent network”. Finding them is difficult, as many hyperparameters are highly correlated to the architecture and to the dataset. We leverage silent networks by designing a spike-based early stopping criterion to accelerate the optimization process of SNNs trained by spike timing dependent plasticity and surrogate gradient. We parallelized the optimization algorithm asynchronously, and ran large-scale experiments on heterogeneous multi-GPU Petascale architecture. Results show that by considering silent networks, we can design more flexible high-dimensional search spaces while maintaining a good efficacy. The optimization algorithm was able to focus on networks with high performances by preventing costly and worthless computation of silent networks.

A fast dual-module hybrid high-dimensional feature selection algorithm

When dealing with large-scale datasets, high-dimensional feature selection plays a crucial role in improving the performance and interpretability of machine learning models. However, it still faces the problems of the “dimensionality curse” and high computational cost in dealing with high-dimensional datasets. In this paper, we develop a dual-module hybrid feature selection algorithm based on correlation filtering and a multi-objective evolutionary algorithm based on dynamic variation distance (HMOFS-CFDVD), which aims to reduce the computational cost of high-dimensional features and search space. Firstly, a coarse-grained feature filtering method based on correlation is proposed, enabling the algorithm to quickly identify potentially better feature subsets in the later stages. After that, a fine-grained feature selection is further implemented based on the multi-objective evolutionary algorithm with sample variation distance to obtain a high-quality feature subset. This stage incorporates a novel population initialization method with self-regulation, an adaptive evolution strategy, an optimal sample selection mechanism based on dynamic sample distance, and an improved diversity mechanism to enhance the evolutionary performance. Moreover, the feature relevance metric is introduced as a third objective to improve the overall algorithm's performance. Experimental results on 12 high-dimensional feature datasets demonstrate that the proposed HMOFS-CFDVD achieves high accuracy and produces a smaller subset of features.

Ab Initio-Based Bond Order Potential for Arsenene Polymorphs Developed via Hierarchical Reinforcement Learning.

Arsenene, a less-explored two-dimensional material, holds the potential for applications in wearable electronics, memory devices, and quantum systems. This study introduces a bond-order potential model with Tersoff formalism, the ML-Tersoff, which leverages multireward hierarchical reinforcement learning (RL), trained on an ab initio data set. This data set covers a spectrum of properties for arsenene polymorphs, enhancing our understanding of its mechanical and thermal behaviors without the complexities of traditional models requiring multiple parameter sets. Our RL strategy utilizes decision trees coupled with a hierarchical reward strategy to accelerate convergence in high-dimensional continuous search spaces. Unlike the Stillinger-Weber approach, which demands separate formalisms for buckled and puckered forms, the ML-Tersoff model concurrently captures multiple properties of the two polymorphs by effectively representing the local environment, thereby avoiding the need for different atomic types. We apply the ML model to understand the mechanical and thermal properties of the arsenene polymorphs and nanostructures. We observe an inverse relationship between the critical strain and temperature in arsenene. Thermal conductivity calculations in nanosheets show good agreement with ab initio data, reflecting a decrease in thermal conductivity attributable to increased anharmonic effects at higher temperatures. We also apply the model to predict the thermal behavior of arsenene nanotubes.

DET-LSH: A Locality-Sensitive Hashing Scheme with Dynamic Encoding Tree for Approximate Nearest Neighbor Search

Locality-sensitive hashing (LSH) is a well-known solution for approximate nearest neighbor (ANN) search in high-dimensional spaces due to its robust theoretical guarantee on query accuracy. Traditional LSH-based methods mainly focus on improving the efficiency and accuracy of the query phase by designing different query strategies, but pay little attention to improving the efficiency of the indexing phase. They typically fine-tune existing data-oriented partitioning trees to index data points and support their query strategies. However, their strategy to directly partition the multi-dimensional space is time-consuming, and performance degrades as the space dimensionality increases. In this paper, we design an encoding-based tree called Dynamic Encoding Tree (DE-Tree) to improve the indexing efficiency and support efficient range queries based on Euclidean distance. Based on DE-Tree, we propose a novel LSH scheme called DET-LSH. DET-LSH adopts a novel query strategy, which performs range queries in multiple independent index DE-Trees to reduce the probability of missing exact NN points, thereby improving the query accuracy. Our theoretical studies show that DET-LSH enjoys probabilistic guarantees on query accuracy. Extensive experiments on real-world datasets demonstrate the superiority of DET-LSH over the state-of-the-art LSH-based methods on both efficiency and accuracy. While achieving better query accuracy than competitors, DET-LSH achieves up to 6x speedup in indexing time and 2x speedup in query time over the state-of-the-art LSH-based methods.

Terminal Trajectory Planning for Synthetic Aperture Radar Imaging Guidance Based on Chronological Iterative Search Framework.

Synthetic aperture radar (SAR) is capable of obtaining the high-resolution 2-D image of the interested target scene, which enables advanced remote sensing and military applications, such as missile terminal guidance. In this article, the terminal trajectory planning for SAR imaging guidance is first investigated. It is found that the guidance performance of an attack platform is determined by the adopted terminal trajectory. Therefore, the aim of the terminal trajectory planning is to generate a set of feasible flight paths to guide the attack platform toward the target and meanwhile obtain the optimized SAR imaging performance for enhanced guidance precision. The trajectory planning is then modeled as a constrained multiobjective optimization problem given a high-dimensional search space, where the trajectory control and SAR imaging performance are comprehensively considered. By utilizing the temporal-order-dependent property of the trajectory planning problem, a chronological iterative search framework (CISF) is proposed. The problem is decomposed into a series of subproblems, where the search space, objective functions, and constraints are reformulated in chronological order. The difficulty of solving the trajectory planning problem is thus significantly alleviated. Then, the search strategy of CISF is devised to solve the subproblems successively. The optimization results of the preceding subproblem can be utilized as the initial input of the subsequent subproblems to enhance the convergence and search performance. Finally, a trajectory planning method is put forward based on CISF. Experimental studies demonstrate the effectiveness and superiority of the proposed CISF compared with the state-of-the-art multiobjective evolutionary methods. The proposed trajectory planning method can generate a set of feasible terminal trajectories with optimized mission performance.

A gray box framework that optimizes a white box logical model using a black box optimizer for simulating cellular responses to perturbations

Predicting cellular responses to perturbations requires interpretable insights into molecular regulatory dynamics to perform reliable cell fate control, despite the confounding non-linearity of the underlying interactions. There is a growing interest in developing machine learning-based perturbation response prediction models to handle the non-linearity of perturbation data, but their interpretation in terms of molecular regulatory dynamics remains a challenge. Alternatively, for meaningful biological interpretation, logical network models such as Boolean networks are widely used in systems biology to represent intracellular molecular regulation. However, determining the appropriate regulatory logic of large-scale networks remains an obstacle due to the high-dimensional and discontinuous search space. To tackle these challenges, we present a scalable derivative-free optimizer trained by meta-reinforcement learning for Boolean network models. The logical network model optimized by the trained optimizer successfully predicts anti-cancer drug responses of cancer cell lines, while simultaneously providing insight into their underlying molecular regulatory mechanisms.

Local Bayesian optimization for controller tuning with crash constraints

Abstract Controller tuning is crucial for closed-loop performance but often involves manual adjustments. Although Bayesian optimization (BO) has been established as a data-efficient method for automated tuning, applying it to large and high-dimensional search spaces remains challenging. We extend a recently proposed local variant of BO to include crash constraints, where the controller can only be successfully evaluated in an a-priori unknown feasible region. We demonstrate the efficiency of the proposed method through simulations and hardware experiments. Our findings showcase the potential of local BO to enhance controller performance and reduce the time and resources necessary for tuning.

FLEX: A fast and light-weight learned index for kNN search in high-dimensional space

The k Nearest Neighbors (kNN) search in high-dimensional space is a fundamental problem with various applications. In this paper, we try to solve this problem by using deep neural networks (DNNs). We apply DNNs to represent complex correlations between high-dimensional objects, enabling us to project similar objects into the same class, thus reducing the search cost. Based on DNNs, we propose two novel techniques to improve query efficiency while keeping accuracy. First, traditional DNNs typically demand extensive training data for achieving high accuracy. To decrease the training size, we design a multi-module DNN framework comprising several small modules. Each module learns to capture part of knowledge for the given query. The collective output of these sub-modules is then seamlessly integrated to form the final result. Second, with machine learning models, the size of candidates are unbounded. Thus, we design a linear-time data layout refinement algorithm, aiming to limit the number of candidates to a small constant. Empirically we find that our approach significantly outperforms the state-of-the-art methods in terms of both time efficiency and space efficiency while still attaining comparable or better accuracy.

Hybrid whale algorithm with evolutionary strategies and filtering for high-dimensional optimization: Application to microarray cancer data.

The standard whale algorithm is prone to suboptimal results and inefficiencies in high-dimensional search spaces. Therefore, examining the whale optimization algorithm components is critical. The computer-generated initial populations often exhibit an uneven distribution in the solution space, leading to low diversity. We propose a fusion of this algorithm with a discrete recombinant evolutionary strategy to enhance initialization diversity. We conduct simulation experiments and compare the proposed algorithm with the original WOA on thirteen benchmark test functions. Simulation experiments on unimodal or multimodal benchmarks verified the better performance of the proposed RESHWOA, such as accuracy, minimum mean, and low standard deviation rate. Furthermore, we performed two data reduction techniques, Bhattacharya distance and signal-to-noise ratio. Support Vector Machine (SVM) excels in dealing with high-dimensional datasets and numerical features. When users optimize the parameters, they can significantly improve the SVM's performance, even though it already works well with its default settings. We applied RESHWOA and WOA methods on six microarray cancer datasets to optimize the SVM parameters. The exhaustive examination and detailed results demonstrate that the new structure has addressed WOA's main shortcomings. We conclude that the proposed RESHWOA performed significantly better than the WOA.

Automated Equations of State Tuning Workflow Using Global Optimization and Physical Constraints

A computational model that can accurately describe the thermodynamics of a hydrocarbon system and its properties under various conditions is a prerequisite for running reservoir and pipeline simulations. Cubic Equations of State (EoS) are mathematical tools used to model the phase and volumetric behavior of reservoir fluids when compositional effects need to be considered. To anticipate uncertainty and enhance the quality of their predictions, EoS models must be adjusted to adequately match the available lab-measured PVT values. This task is challenging given that there are many potential tuning parameters, thus leading to various tuning results of questionable validity. In this paper, we present an automated EoS tuning workflow that employs a Generalized Pattern Search (GPS) optimizer for efficient tuning of a cubic EoS model. Specifically, we focus on the Peng–Robinson (PR) model, which is the oil and gas industry standard, to accurately capture the behavior of diverse multicomponent, complex hydrocarbon mixtures encountered in subsurface reservoirs. This approach surpasses the limitations of conventional gradient-based (GB) methods, which are susceptible to getting trapped in local optima. The proposed technique also allows physical constraints to be imposed on the optimization procedure. A gas condensate and an H2S-rich oil were used to demonstrate the effectiveness of the GPS algorithm in finding an optimized solution for high-dimensional search spaces, and its superiority over conventional gradient-based optimization was confirmed by automatically tracking globally optimal and physically sound solutions.

A co-evolutionary algorithm based on sparsity clustering for sparse large-scale multi-objective optimization

Sparse large-scale multi-objective optimization problems (LSMOPs), which are characterized by high dimensional search space and sparse Pareto optimal solutions, have a widespread existence in academic research and practical applications. While the high dimensional decision space poses challenges to multi-objective evolutionary algorithms (MOEAs), the difficulty of solving sparse LSMOPs can be alleviated by utilizing the prior knowledge that the optimal solutions are sparse. In this paper, a co-evolutionary algorithm based on sparsity clustering, namely SCEA, is proposed, where the prior knowledge of sparse optimal solutions is utilized explicitly. At each generation, SCEA first calculates the current optimal sparsity by sparsity clustering. Then, SCEA divides the population into a winner subpopulation and two loser subpopulations. While the winner subpopulation reproduces offspring solutions by conventional genetic operators, the loser subpopulations generate offspring solutions along two competitive directions under the guidance of current optimal sparsity and variable importance. In the experiments, four state-of-the-art MOEAs are selected as the comparative algorithms. Experimental results show that the proposed algorithm is superior to the four competitors on both benchmark problems and practical applications, which include the sparse signal reconstruction problem, the community detection problem, and the instance selection problem.

Evolutionary multiobjective optimization assisted by scalarization function approximation for high-dimensional expensive problems

Surrogate-assisted evolutionary algorithms (SAEAs) are a promising approach for solving expensive multiobjective optimization problems, but they often cannot address high-dimensional problems. Although one common approach to overcoming this challenge is to construct reliable surrogates, their accuracy inevitably deteriorates in a high-dimensional search space. Thus, this paper presents a novel SAEA based on scalarization function approximation, which is designed to strengthen its robustness against this deterioration. The proposed algorithm constructs an approximation model for each scalarization function defined in a decomposition-based framework. Each decomposed problem is then solved using multiple independent models trained for its neighbor problems. The intent is to decrease the risk of search performance degradations caused by unreliable approximations and retain the redundancy of the surrogate-assisted search to hedge the risk of over-fitting. Furthermore, each approximation model is adapted to a promising region of its corresponding decomposed problem to reduce the complexity of model fitting given a limited number of training samples. Experimental results show that the proposed algorithm is competitive with state-of-the-art SAEAs adapted for high-dimensional problems.

Inverse design of a Mamyshev oscillator with MW peak power by a particle swarm optimization algorithm

In this study we demonstrate implementation of a particle swarm optimization algorithm for inverse design of a fiber Mamyshev oscillator. We perform an extensive search in high-dimensional space of cavity parameters to maximize the output pulses’ peak power after external compression. The best obtained pulses have peak power exceeding 2 MW, energy >140nJ, and duration <40fs, and exhibit capabilities of handling ∼75π nonlinear phase accumulation.

Efficient and Robust Point Cloud Registration via Heuristics-guided Parameter Search.

Estimating the rigid transformation with 6 degrees of freedom based on a putative 3D correspondence set is a crucial procedure in point cloud registration. Existing correspondence identification methods usually lead to large outlier ratios(> 95% is common), underscoring the significance of robust registration methods. Many researchers turn to parameter search-based strategies(e.g., Branch-and-Bround) for robust registration. Although related methods show high robustness, their efficiency is limited to the high-dimensional search space. This paper proposes a heuristics-guided parameter search strategy to accelerate the search while maintaining high robustness. We first sample some correspondences(i.e., heuristics) and then just need to sequentially search the feasible regions that make each sample an inlier. Our strategy largely reduces the search space and can guarantee accuracy with only a few inlier samples, therefore enjoying an excellent trade-off between efficiency and robustness. Since directly parameterizing the 6-dimensional nonlinear feasible region for efficient search is intractable, we construct a three-stage decomposition pipeline to reparameterize the feasible region, resulting in three lower-dimensional sub-problems that are easily solvable via our strategy. Besides reducing the searching dimension, our decomposition enables the leverage of 1-dimensional interval stabbing at all three stages for searching acceleration. Moreover, we propose a valid sampling strategy to guarantee our sampling effectiveness, and a compatibility verification setup to further accelerate our search. Extensive experiments on both simulated and real-world datasets demonstrate that our approach exhibits comparable robustness with state-of-the-art methods while achieving a significant efficiency boost.

Dynamic Population on Bio-Inspired Algorithms Using Machine Learning for Global Optimization.

In the optimization field, the ability to efficiently tackle complex and high-dimensional problems remains a persistent challenge. Metaheuristic algorithms, with a particular emphasis on their autonomous variants, are emerging as promising tools to overcome this challenge. The term "autonomous" refers to these variants' ability to dynamically adjust certain parameters based on their own outcomes, without external intervention. The objective is to leverage the advantages and characteristics of an unsupervised machine learning clustering technique to configure the population parameter with autonomous behavior, and emphasize how we incorporate the characteristics of search space clustering to enhance the intensification and diversification of the metaheuristic. This allows dynamic adjustments based on its own outcomes, whether by increasing or decreasing the population in response to the need for diversification or intensification of solutions. In this manner, it aims to imbue the metaheuristic with features for a broader search of solutions that can yield superior results. This study provides an in-depth examination of autonomous metaheuristic algorithms, including Autonomous Particle Swarm Optimization, Autonomous Cuckoo Search Algorithm, and Autonomous Bat Algorithm. We submit these algorithms to a thorough evaluation against their original counterparts using high-density functions from the well-known CEC LSGO benchmark suite. Quantitative results revealed performance enhancements in the autonomous versions, with Autonomous Particle Swarm Optimization consistently outperforming its peers in achieving optimal minimum values. Autonomous Cuckoo Search Algorithm and Autonomous Bat Algorithm also demonstrated noteworthy advancements over their traditional counterparts. A salient feature of these algorithms is the continuous nature of their population, which significantly bolsters their capability to navigate complex and high-dimensional search spaces. However, like all methodologies, there were challenges in ensuring consistent performance across all test scenarios. The intrinsic adaptability and autonomous decision making embedded within these algorithms herald a new era of optimization tools suited for complex real-world challenges. In sum, this research accentuates the potential of autonomous metaheuristics in the optimization arena, laying the groundwork for their expanded application across diverse challenges and domains. We recommend further explorations and adaptations of these autonomous algorithms to fully harness their potential.

Maximizing Performance and Stability of Organic Solar Cells at Low Driving Force for Charge Separation.

Thanks to the development of novel electron acceptor materials, the power conversion efficiencies (PCE) of organic photovoltaic (OPV) devices are now approaching 20%. Further improvement of PCE is complicated by the need for a driving force to split strongly bound excitons into free charges, causing voltage losses. This review discusses recent approaches to finding efficient OPV systems with minimal driving force, combining near unity quantum efficiency (maximum short circuit currents) with optimal energy efficiency (maximum open circuit voltages). The authors discuss apparently contradicting results on the amount of exciton binding in recent literature, and approaches to harmonize the findings. A comprehensive view is then presented on motifs providing a driving force for charge separation, namely hybridization at the donor:acceptor interface and polarization effects in the bulk, of which quadrupole moments (electrostatics) play a leading role. Apart from controlling the energies of the involved states, these motifs also control the dynamics of recombination processes, which are essential to avoid voltage and fill factor losses. Importantly, all motifs are shown to depend on both molecular structure and process conditions. The resulting high dimensional search space advocates for high throughput (HT) workflows. The final part of the review presents recent HT studies finding consolidated structure-property relationships in OPV films and devices from various deposition methods, from research to industrial upscaling.

Data-Driven Discovery of Lithium-Ion Battery State of Charge Dynamics

Abstract We present a physics-inspired input/output predictor of lithium-ion batteries (LiBs) for online state-of-charge (SOC) prediction. The complex electrochemical behavior of batteries results in nonlinear and high-dimensional dynamics. Accurate SOC prediction is paramount for increased performance, improved operational safety, and extended longevity of LiBs. The battery's internal parameters are cell-dependent and change with operating conditions and battery health variations. We present a data-driven solution to discover governing equations pertaining to SOC dynamics from battery operando measurements. Our approach relaxes the need for detailed knowledge of the battery's composition while maintaining prediction fidelity. The predictor consists of a library of candidate terms and a set of coefficients found via a sparsity-promoting algorithm. The library was enhanced with explicit physics-inspired terms to improve the predictor's interpretability and generalizability. Further, we developed a Monte Carlo search of additional nonlinear terms to efficiently explore the high-dimensional search space and improve the characterization of highly nonlinear behaviors. Also, we developed a hyperparameter autotuning approach for identifying optimal coefficients that balance accuracy and complexity. The resulting SOC predictor achieved high predictive performance scores (RMSE) of 2.2×10−6 and 4.8×10−4, respectively, for training and validation on experimental results corresponding to a stochastic drive cycle. Furthermore, the predictor achieved an RMSE of 8.5×10−4 on unseen battery measurements corresponding to the standard US06 drive cycle, further showcasing the adaptability of the predictor and the enhanced modeling approach to new conditions.

Surrogate-assisted evolutionary algorithm with decomposition-based local learning for high-dimensional multi-objective optimization

When the evolutionary algorithm is applied to handle high-dimensional expensive multi-objective optimization problems (MOPs), population evolution is crucial since it controls exploration and exploitation and decides if promising candidate solutions could be generated. However, little attention has been paid to this issue, evolution operators based on Genetic Algorithm (GA) and Differential Evolution (DE) are still the two most common approaches, whose convergence on high-dimensional MOPs with a limited number of fitness evaluations remains challenging. In this paper, we propose a decomposition-based local learning strategy to accelerate convergence in the high-dimensional search space of MOPs. Specifically, an individual is updated by learning from one of the best solutions of its corresponding local area based on the multi-objective decomposition approach. Accordingly, a surrogate-assisted evolutionary algorithm is proposed for better solving expensive high-dimensional MOPs. Experimental studies on MOPs with up to 100 decision variables and with 300 fitness evaluations demonstrate the effectiveness of the proposed method. Furthermore, we use the proposed method to solve a 2-objective shape design problem of the blended wing-body underwater glider (BWBUG) with 39 decision variables, and an impressive solution set is obtained.