High-dimensional Search Space Research Articles

Optimization of high-dimensional expensive multi-objective problems using multi-mode radial basis functions

Numerous surrogate-assisted evolutionary algorithms are developed for multi-objective expensive problems with low dimensions, but scarce works have paid attention to that with high dimensions, i.e., generally more than 30 decision variables. In this paper, we propose a multi-mode radial basis functions-assisted evolutionary algorithm (MMRAEA) for solving high-dimensional expensive multi-objective optimization problems. To improve the reliability, the proposed algorithm uses radial basis functions based on three modes to cooperate to provide the qualities and uncertainty information of candidate solutions. Meanwhile, bi-population based on competitive swarm optimizer and genetic algorithm are applied for better exploration and exploitation in high-dimensional search space. Accordingly, an infill criterion based on multi-mode of radial basis functions that comprehensively considers the quality and uncertainty of candidate solutions is proposed. Experimental results on widely-used benchmark problems with up to 100 decision variables demonstrate the effectiveness of our proposal. Furthermore, the proposed method is applied to the structure optimization of the blended-wing-body underwater glider (BWBUG) and gets impressive solutions.

Ocotillo Optimization Algorithm (OcOA): A Desert-Inspired Metaheuristic for Adaptive Optimization

In this paper, we propose the Ocotillo Optimization Algorithm (OcOA), a novel desert-inspired metaheuristic designed to solve complex optimization problems. Inspired by the adaptive strategies of desert plants, OcOA aims to achieve a balance between exploration and exploitation in high-dimensional and multimodal search spaces. The algorithm dynamically adjusts its behavior based on feedback from prior iterations, optimizing both search breadth and solution refinement. To evaluate its effectiveness, OcOA was tested against several well-known algorithms on a range of benchmark functions, including unimodal and multimodal functions from the CEC 2005 suite such as Sphere, Rosenbrock, Ackley, and Rastrigin. The results demonstrate that OcOA outperforms competing approaches in terms of accuracy, convergence speed, and computational efficiency. Additionally, its adaptability was validated through feature selection tasks, highlighting its robustness in handling both continuous and discrete optimization challenges. This study positions OcOA as a competitive optimization tool for various real-world applications

Enhancing Healthcare Through Metaheuristic Based Deep Learning: Microarray Gene Expression Image Classification Model

Microarray gene expression data analysis has revolutionized genomics by providing insights into the genetic basis of diseases, including cancer. However, the high dimensionality of microarray data poses significant challenges for accurate classification, often leading to overfitting and poor generalization. This study proposes a novel framework that integrates the DNA Bidirectional Encoder Representations from Transformers (DNABERT) with the Mayfly Optimization Algorithm (MFO) to improve the classification accuracy of microarray gene expression data. DNABERT, a transformer-based model pretrained on genomic sequences, excels in learning complex gene interactions through bidirectional contextual embeddings. MFO, a bio-inspired optimization technique, addresses feature selection and hyperparameter tuning by balancing exploration and exploitation in high-dimensional search spaces. The framework, DNABERT-MFO, leverages MFO to identify the most relevant gene subsets and optimize DNABERT's hyperparameters, improving the model’s performance. Evaluations conducted on multiple microarray datasets, demonstrate that DNABERT-MFO with overall accuracy of above 80% significantly outperforms traditional machine learning methods and standalone deep learning models in classification accuracy. This integrated method not only enhances the robustness of gene expression data analysis but also offers a powerful tool for research and clinical applications in genomics. The proposed method addresses key limitations of existing techniques and provides a promising avenue for future advancements in gene expression classification.

High-dimensional expensive optimization by Kriging-assisted multiobjective evolutionary algorithm with dimensionality reduction

Surrogate-assisted multi-objective evolutionary algorithms (SA-MOEAs) have made significant progress in solving expensive multi- and many-objective optimization problems. However, most of them perform well in low-dimensional settings but often struggle with high-dimensional problems. The main reason is that some techniques used in SA-MOEAs, like the Kriging model, are ineffective in exploring high-dimensional search spaces. As a result, this research investigates frameworks incorporating dimensionality reduction techniques to conduct modeling and optimization tasks on dimensionality reduction decision spaces. This article uses a singular value decomposition method to map the high-dimensional decision space into a low-dimensional one, then employs a feature fusion strategy to combine low-dimensional features with high-dimensional ones for better representation. Subsequently, these low-dimensional features are used to train the Kriging-based surrogates to select promising solutions within a limited number of function evaluations. In addition, this article provides two types of evolutionary modes to balance exploration and exploitation. Experimental results demonstrate the effectiveness of the proposed SA-MOEA compared to several state-of-the-art algorithms.

Efficient and Robust Point Cloud Registration via Heuristics-Guided Parameter Search.

Estimating the rigid transformation with 6 degrees of freedom based on a putative 3D correspondence set is a crucial procedure in point cloud registration. Existing correspondence identification methods usually lead to large outlier ratios (>95% is common), underscoring the significance of robust registration methods. Many researchers turn to parameter search-based strategies (e.g., Branch-and-Bround) for robust registration. Although related methods show high robustness, their efficiency is limited to the high-dimensional search space. This paper proposes a heuristics-guided parameter search strategy to accelerate the search while maintaining high robustness. We first sample some correspondences (i.e., heuristics) and then just need to sequentially search the feasible regions that make each sample an inlier. Our strategy largely reduces the search space and can guarantee accuracy with only a few inlier samples, therefore enjoying an excellent trade-off between efficiency and robustness. Since directly parameterizing the 6-dimensional nonlinear feasible region for efficient search is intractable, we construct a three-stage decomposition pipeline to reparameterize the feasible region, resulting in three lower-dimensional sub-problems that are easily solvable via our strategy. Besides reducing the searching dimension, our decomposition enables the leverage of 1-dimensional interval stabbing at all three stages for searching acceleration. Moreover, we propose a valid sampling strategy to guarantee our sampling effectiveness, and a compatibility verification setup to further accelerate our search. Extensive experiments on both simulated and real-world datasets demonstrate that our approach exhibits comparable robustness with state-of-the-art methods while achieving a significant efficiency boost.

Inverse design of high-strength medium-Mn steel using a machine learning-aided genetic algorithm approach

To develop medium-Mn steels with an ultimate tensile strength (UTS) exceeding 2 GPa and excellent ductility, we created a highly accurate UTS prediction machine learning (ML) model using a boosted decision tree model and 1520 dataset of tensile properties of medium-Mn steels with micro-alloying elements. We also optimized the hyper-parameters of a genetic algorithm (GA) using the Shannon diversity index to enhance search efficiency while retaining diversity. In a high-dimensional search space with millions of potential combinations, the ML-GA approach efficiently identified diverse chemical compositions and austenitizing conditions to achieve UTSs above 2 GPa. The k-means clustering method then grouped them into five distinct specimens based on similarities. These five specimens, fabricated using inversly designed chemical compositions and austenitizing temperatures, successfully exhibited UTSs exceeding 2 GPa and greater ductility compared to hot-stamped C steels. These excellent tensile properties were attributed to grain refinement resulting from the low austenitizing temperature and the pinning effect of micro-alloying element carbides, such as TiC and VC.

Navigating the complexity of hybrid materials without structural dependency: PerovGNN as a map

Hybrid organic-inorganic materials, spearheaded by perovskites, have showcased exceptional potential in optical and electrical applications. However, finding suitable compositions with desired properties is challenging due to the vast, high-dimensional search space. Traditional machine learning techniques often fall short in capturing crucial interactions within the composites of hybrid organic-inorganic materials, while others necessitate structural information that is often unavailable prior to material synthesis. To address this, we propose PerovGNN, a structure-agnostic graph representation learning method for property prediction. It transforms the chemical formula of each mixed perovskite into a weighted graph of atoms, with weights representing the relative frequency of interactions between neighboring atoms. By embedding both fractional ratios and atomic features, this graph formulation effectively retains crucial mixture information and learns a superior representation required for a specific targeted property. We applied PerovGNN to experimental datasets for mixed HOIPs bandgap prediction, achieving a lower mean absolute error (MAE) of 0.0483 eV compared to other methods. New perovskites with bandgaps ranging from 1.3 to 2.0 eV were experimentally characterized, yielding a consistent MAE of 0.043 eV. Our work presents a unified model for predicting properties of inorganic, organic, and hybrid crystals, serving as a map for researchers navigating vast design spaces.

Dynamical Sphere Regrouping Particle Swarm Optimization Programming: An Automatic Programming Algorithm Avoiding Premature Convergence

Symbolic regression plays a crucial role in machine learning and data science by allowing the extraction of meaningful mathematical models directly from data without imposing a specific structure. This level of adaptability is especially beneficial in scientific and engineering fields, where comprehending and articulating the underlying data relationships is just as important as making accurate predictions. Genetic Programming (GP) has been extensively utilized for symbolic regression and has demonstrated remarkable success in diverse domains. However, GP’s heavy reliance on evolutionary mechanisms makes it computationally intensive and challenging to handle. On the other hand, Particle Swarm Optimization (PSO) has demonstrated remarkable performance in numerical optimization with parallelism, simplicity, and rapid convergence. These attributes position PSO as a compelling option for Automatic Programming (AP), which focuses on the automatic generation of programs or mathematical models. Particle Swarm Programming (PSP) has emerged as an alternative to Genetic Programming (GP), with a specific emphasis on harnessing the efficiency of PSO for symbolic regression. However, PSP remains unsolved due to the high-dimensional search spaces and local optimal regions in AP, where traditional PSO can encounter issues such as premature convergence and stagnation. To tackle these challenges, we introduce Dynamical Sphere Regrouping PSO Programming (DSRegPSOP), an innovative PSP implementation that integrates DSRegPSO’s dynamical sphere regrouping and momentum conservation mechanisms. DSRegPSOP is specifically developed to deal with large-scale, high-dimensional search spaces featuring numerous local optima, thus proving effective behavior for symbolic regression tasks. We assess DSRegPSOP by generating 10 mathematical expressions for mapping points from functions with varying complexity, including noise in position and cost evaluation. Moreover, we also evaluate its performance using real-world datasets. Our results show that DSRegPSOP effectively addresses the shortcomings of PSO in PSP by producing mathematical models entirely generated by AP that achieve accuracy similar to other machine learning algorithms optimized for regression tasks involving numerical structures. Additionally, DSRegPSOP combines the benefits of symbolic regression with the efficiency of PSO.

Synergizing Quality-Diversity with Descriptor-Conditioned Reinforcement Learning

A hallmark of intelligence is the ability to exhibit a wide range of effective behaviors. Inspired by this principle, Quality-Diversity algorithms, such as, are evolutionary methods designed to generate a set of diverse and high-fitness solutions. However, as a genetic algorithm, relies on random mutations, which can become inefficient in high-dimensional search spaces, thus limiting its scalability to more complex domains, such as learning to control agents directly from high-dimensional inputs. To address this limitation, advanced methods like and have been developed, which combine actor-critic techniques from Reinforcement Learning with, significantly enhancing the performance and efficiency of Quality-Diversity algorithms in complex, high-dimensional tasks. While these methods have successfully leveraged the trained critic to guide more effective mutations, the potential of the trained actor remains underutilized in improving both the quality and diversity of the evolved population. In this work, we introduce, an extension of that utilizes the descriptor-conditioned actor as a generative model to produce diverse solutions, which are then injected into the offspring batch at each generation. Additionally, we present an empirical analysis of the fitness and descriptor reproducibility of the solutions discovered by each algorithm. Finally, we present a second empirical analysis shedding light on the synergies between the different variations operators and explaining the performance improvement from to.

Ultrafast artificial intelligence: machine learning with atomic-scale quantum systems

Abstract We train a model atom to recognize hand-written digits in the range 0-9, employing intense light--matter interaction as a computational resource. For training, the images of the digits are converted into shaped laser pulses (data input pulses). Simultaneously with an input pulse, another shaped pulse (program pulse), polarized in the orthogonal direction, is applied to the atom and the system evolves quantum mechanically according to the time-dependent Schr"odinger equation. The purpose of the optimal program pulse is to direct the system into specific atomic final states (classification states) that correspond to the input digits. 
A success rate of about 40\% is achieved when using a basic optimization scheme that might be limited by the computational resources for finding the optimal program pulse in a high-dimensional search space. Our key result is the capability of the laser-programmed atom to generalize, i.e.,~the classification of unseen images is improved by training. This atom-sized machine-learning image-recognition scheme operates on time scales down to tens of femtoseconds, is scalable towards larger (e.g. molecular) systems, and is readily reprogrammable towards other learning/classification tasks.

A comprehensive learning based swarm optimization approach for feature selection in gene expression data

Gene expression data analysis is challenging due to the high dimensionality and complexity of the data. Feature selection, which identifies relevant genes, is a common preprocessing step. We propose a Comprehensive Learning-Based Swarm Optimization (CLBSO) approach for feature selection in gene expression data. CLBSO leverages the strengths of ants and grasshoppers to efficiently explore the high-dimensional search space. Ants perform local search and leave pheromone trails to guide the swarm, while grasshoppers use their ability to jump long distances to explore new regions and avoid local optima. The proposed approach was evaluated on several publicly available gene expression datasets and compared with state-of-the-art feature selection methods. CLBSO achieved an average accuracy improvement of 15% over the original high-dimensional data and outperformed other feature selection methods by up to 10%. For instance, in the Pancreatic cancer dataset, CLBSO achieved 97.2% accuracy, significantly higher than XGBoost-MOGA's 84.0%. Convergence analysis showed CLBSO required fewer iterations to reach optimal solutions. Statistical analysis confirmed significant performance improvements, and stability analysis demonstrated consistent gene subset selection across different runs. These findings highlight the robustness and efficacy of CLBSO in handling complex gene expression datasets, making it a valuable tool for enhancing classification tasks in bioinformatics.

Parallel hyperparameter optimization of spiking neural networks

Hyperparameter optimization of spiking neural networks (SNNs) is a difficult task which has not yet been deeply investigated in the literature. In this work, we designed a scalable constrained Bayesian based optimization algorithm that prevents sampling in non-spiking areas of an efficient high dimensional search space. These search spaces contain infeasible solutions that output no or only a few spikes during the training or testing phases, we call such a mode a “silent network”. Finding them is difficult, as many hyperparameters are highly correlated to the architecture and to the dataset. We leverage silent networks by designing a spike-based early stopping criterion to accelerate the optimization process of SNNs trained by spike timing dependent plasticity and surrogate gradient. We parallelized the optimization algorithm asynchronously, and ran large-scale experiments on heterogeneous multi-GPU Petascale architecture. Results show that by considering silent networks, we can design more flexible high-dimensional search spaces while maintaining a good efficacy. The optimization algorithm was able to focus on networks with high performances by preventing costly and worthless computation of silent networks.

A fast dual-module hybrid high-dimensional feature selection algorithm

When dealing with large-scale datasets, high-dimensional feature selection plays a crucial role in improving the performance and interpretability of machine learning models. However, it still faces the problems of the “dimensionality curse” and high computational cost in dealing with high-dimensional datasets. In this paper, we develop a dual-module hybrid feature selection algorithm based on correlation filtering and a multi-objective evolutionary algorithm based on dynamic variation distance (HMOFS-CFDVD), which aims to reduce the computational cost of high-dimensional features and search space. Firstly, a coarse-grained feature filtering method based on correlation is proposed, enabling the algorithm to quickly identify potentially better feature subsets in the later stages. After that, a fine-grained feature selection is further implemented based on the multi-objective evolutionary algorithm with sample variation distance to obtain a high-quality feature subset. This stage incorporates a novel population initialization method with self-regulation, an adaptive evolution strategy, an optimal sample selection mechanism based on dynamic sample distance, and an improved diversity mechanism to enhance the evolutionary performance. Moreover, the feature relevance metric is introduced as a third objective to improve the overall algorithm's performance. Experimental results on 12 high-dimensional feature datasets demonstrate that the proposed HMOFS-CFDVD achieves high accuracy and produces a smaller subset of features.

Adaptive Meta-heuristic Framework for Real-time Dynamic Obstacle Avoidance in Redundant Robot Manipulators

Robotic manipulator faces a challenge in navigating dynamic environments while ensuring collision-free trajectories, especially for redundant manipulators. Inverse kinematics involves finding joint angles to reach a specific point in 3D space. The shift from classical analytical and numerical methods to optimization heuristic algorithms is driven by the increasing complexity of robotic systems and the demand for more versatile and adaptive solutions. Meta-heuristic algorithms offer a transformative approach by framing the inverse kinematics problem as an optimization challenge, providing a flexible and robust means to navigate complex solution spaces. Metaheuristic algorithms, known for their ability to explore high-dimensional search spaces and avoid local optima, offer robust solutions for these challenges. They enhance computational efficiency, enabling real-time decision-making and obstacle avoidance, making them ideal for complex robotic applications. These characteristics of the metaheuristic algorithms can used in developing an integrated framework that offers complete solution to robot manipulators. This research article presents a generalized framework leveraging meta-heuristic algorithms to address dynamic obstacle avoidance in redundant manipulators. The framework uses meta-heuristic algorithms as the inverse kinematics solver, 3D trajectory planner, and obstacle avoidance mechanism, encompassing both static and dynamic obstacles. The proposed framework is generalized and gives the user to select the type of robot manipulator, with any number of links with any custom trajectory within the workspace of the robot manipulator. Also, any metaheuristic algorithm can be used in the proposed framework. The proposed framework is implemented in MATLAB’s app designer for simulation with six different meta-heuristic algorithms. The effectiveness of the framework was evaluated in terms of its capability to generate 3d path, its ability to follow generated trajectory, while seamlessly adapting to dynamically changing environments. Through simulation, the framework showcased robust performance in navigating workspaces with moving obstacles, ensuring collision-free motion for redundant manipulators.

Ab Initio-Based Bond Order Potential for Arsenene Polymorphs Developed via Hierarchical Reinforcement Learning.

Arsenene, a less-explored two-dimensional material, holds the potential for applications in wearable electronics, memory devices, and quantum systems. This study introduces a bond-order potential model with Tersoff formalism, the ML-Tersoff, which leverages multireward hierarchical reinforcement learning (RL), trained on an ab initio data set. This data set covers a spectrum of properties for arsenene polymorphs, enhancing our understanding of its mechanical and thermal behaviors without the complexities of traditional models requiring multiple parameter sets. Our RL strategy utilizes decision trees coupled with a hierarchical reward strategy to accelerate convergence in high-dimensional continuous search spaces. Unlike the Stillinger-Weber approach, which demands separate formalisms for buckled and puckered forms, the ML-Tersoff model concurrently captures multiple properties of the two polymorphs by effectively representing the local environment, thereby avoiding the need for different atomic types. We apply the ML model to understand the mechanical and thermal properties of the arsenene polymorphs and nanostructures. We observe an inverse relationship between the critical strain and temperature in arsenene. Thermal conductivity calculations in nanosheets show good agreement with ab initio data, reflecting a decrease in thermal conductivity attributable to increased anharmonic effects at higher temperatures. We also apply the model to predict the thermal behavior of arsenene nanotubes.

DET-LSH: A Locality-Sensitive Hashing Scheme with Dynamic Encoding Tree for Approximate Nearest Neighbor Search

Locality-sensitive hashing (LSH) is a well-known solution for approximate nearest neighbor (ANN) search in high-dimensional spaces due to its robust theoretical guarantee on query accuracy. Traditional LSH-based methods mainly focus on improving the efficiency and accuracy of the query phase by designing different query strategies, but pay little attention to improving the efficiency of the indexing phase. They typically fine-tune existing data-oriented partitioning trees to index data points and support their query strategies. However, their strategy to directly partition the multi-dimensional space is time-consuming, and performance degrades as the space dimensionality increases. In this paper, we design an encoding-based tree called Dynamic Encoding Tree (DE-Tree) to improve the indexing efficiency and support efficient range queries based on Euclidean distance. Based on DE-Tree, we propose a novel LSH scheme called DET-LSH. DET-LSH adopts a novel query strategy, which performs range queries in multiple independent index DE-Trees to reduce the probability of missing exact NN points, thereby improving the query accuracy. Our theoretical studies show that DET-LSH enjoys probabilistic guarantees on query accuracy. Extensive experiments on real-world datasets demonstrate the superiority of DET-LSH over the state-of-the-art LSH-based methods on both efficiency and accuracy. While achieving better query accuracy than competitors, DET-LSH achieves up to 6x speedup in indexing time and 2x speedup in query time over the state-of-the-art LSH-based methods.

Terminal Trajectory Planning for Synthetic Aperture Radar Imaging Guidance Based on Chronological Iterative Search Framework.

Synthetic aperture radar (SAR) is capable of obtaining the high-resolution 2-D image of the interested target scene, which enables advanced remote sensing and military applications, such as missile terminal guidance. In this article, the terminal trajectory planning for SAR imaging guidance is first investigated. It is found that the guidance performance of an attack platform is determined by the adopted terminal trajectory. Therefore, the aim of the terminal trajectory planning is to generate a set of feasible flight paths to guide the attack platform toward the target and meanwhile obtain the optimized SAR imaging performance for enhanced guidance precision. The trajectory planning is then modeled as a constrained multiobjective optimization problem given a high-dimensional search space, where the trajectory control and SAR imaging performance are comprehensively considered. By utilizing the temporal-order-dependent property of the trajectory planning problem, a chronological iterative search framework (CISF) is proposed. The problem is decomposed into a series of subproblems, where the search space, objective functions, and constraints are reformulated in chronological order. The difficulty of solving the trajectory planning problem is thus significantly alleviated. Then, the search strategy of CISF is devised to solve the subproblems successively. The optimization results of the preceding subproblem can be utilized as the initial input of the subsequent subproblems to enhance the convergence and search performance. Finally, a trajectory planning method is put forward based on CISF. Experimental studies demonstrate the effectiveness and superiority of the proposed CISF compared with the state-of-the-art multiobjective evolutionary methods. The proposed trajectory planning method can generate a set of feasible terminal trajectories with optimized mission performance.

A gray box framework that optimizes a white box logical model using a black box optimizer for simulating cellular responses to perturbations

Predicting cellular responses to perturbations requires interpretable insights into molecular regulatory dynamics to perform reliable cell fate control, despite the confounding non-linearity of the underlying interactions. There is a growing interest in developing machine learning-based perturbation response prediction models to handle the non-linearity of perturbation data, but their interpretation in terms of molecular regulatory dynamics remains a challenge. Alternatively, for meaningful biological interpretation, logical network models such as Boolean networks are widely used in systems biology to represent intracellular molecular regulation. However, determining the appropriate regulatory logic of large-scale networks remains an obstacle due to the high-dimensional and discontinuous search space. To tackle these challenges, we present a scalable derivative-free optimizer trained by meta-reinforcement learning for Boolean network models. The logical network model optimized by the trained optimizer successfully predicts anti-cancer drug responses of cancer cell lines, while simultaneously providing insight into their underlying molecular regulatory mechanisms.

Local Bayesian optimization for controller tuning with crash constraints

Abstract Controller tuning is crucial for closed-loop performance but often involves manual adjustments. Although Bayesian optimization (BO) has been established as a data-efficient method for automated tuning, applying it to large and high-dimensional search spaces remains challenging. We extend a recently proposed local variant of BO to include crash constraints, where the controller can only be successfully evaluated in an a-priori unknown feasible region. We demonstrate the efficiency of the proposed method through simulations and hardware experiments. Our findings showcase the potential of local BO to enhance controller performance and reduce the time and resources necessary for tuning.

FLEX: A fast and light-weight learned index for kNN search in high-dimensional space

The k Nearest Neighbors (kNN) search in high-dimensional space is a fundamental problem with various applications. In this paper, we try to solve this problem by using deep neural networks (DNNs). We apply DNNs to represent complex correlations between high-dimensional objects, enabling us to project similar objects into the same class, thus reducing the search cost. Based on DNNs, we propose two novel techniques to improve query efficiency while keeping accuracy. First, traditional DNNs typically demand extensive training data for achieving high accuracy. To decrease the training size, we design a multi-module DNN framework comprising several small modules. Each module learns to capture part of knowledge for the given query. The collective output of these sub-modules is then seamlessly integrated to form the final result. Second, with machine learning models, the size of candidates are unbounded. Thus, we design a linear-time data layout refinement algorithm, aiming to limit the number of candidates to a small constant. Empirically we find that our approach significantly outperforms the state-of-the-art methods in terms of both time efficiency and space efficiency while still attaining comparable or better accuracy.