Exploration Algorithm Research Articles

Distributed exploration algorithm for GPS-denied areas based on flocking rules

This paper addresses the problem of having a multi-agent system (MAS) search for areas of higher relative importance as measured by an unknown utility function. In the envisioned scenario, there are inexpensive agents without localisation sensors and limited communication capabilities. More expensive nodes serve as fixed towers and forward noisy position and velocity measurements using directional antennae. There is no assumptions on initial network connectivity. By proposing a set of flocking rules and set-membership estimation, the formation drives to a vicinity of nearby local maximum of the function while having theoretical guarantees of no collisions. The performance of the method is evaluated both in Matlab simulations and using the Crazyswarm package under the robot operating system (ROS) environment, including cases of moving destinations, obstacles, undesirable zones, and different number of nodes and sizes of the mission plane.

Chemical Reaction Networks from Scratch with Reaction Prediction and Kinetics-Guided Exploration.

Algorithmic reaction explorations based on transition state searches can now routinely predict relatively short reaction sequences involving small molecules. However, applying these algorithms to deeper chemical reaction network (CRN) exploration still requires the development of more efficient and accurate exploration policies. Here, an exploration algorithm, which we name yet another kinetic strategy (YAKS), is demonstrated that uses microkinetic simulations of the nascent network to achieve cost-effective, deep network exploration. Key features of the algorithm are the automatic incorporation of bimolecular reactions between network intermediates, compatibility with short-lived but kinetically important species, and incorporation of rate uncertainty into the exploration policy. In validation case studies of glucose pyrolysis, the algorithm rediscovers reaction pathways previously discovered by heuristic exploration policies and elucidates new reaction pathways for experimentally obtained products. The resulting CRN is the first to connect all major experimental pyrolysis products to glucose. Additional case studies are presented that investigate the role of reaction rules, rate uncertainty, and bimolecular reactions. These case studies show that naïve exponential growth estimates can vastly overestimate the actual number of kinetically relevant pathways in the physical reaction networks. In light of this, further improvements in exploration policies and reaction prediction algorithms make it feasible that CRNs might soon be routinely predictable in some contexts.

Research on Traversal Path Planning and Collaborative Scheduling for Corn Harvesting and Transportation in Hilly Areas Based on Dijkstra’s Algorithm and Improved Harris Hawk Optimization

This study addresses the challenges of long traversal paths, low efficiency, high fuel consumption, and costs in the collaborative harvesting of corn by harvesters and grain transport vehicles in hilly areas. A path-planning and collaborative scheduling method is proposed, combining Dijkstra’s algorithm with the Improved Harris Hawk Optimization (IHHO) algorithm. A field model based on Digital Elevation Model (DEM) data is created for full coverage path planning, reducing traversal path length. A field transfer road network is established, and Dijkstra’s algorithm is used to calculate distances between fields. A multi-objective collaborative scheduling model is then developed to minimize fuel consumption, scheduling costs, and time. The IHHO algorithm enhances search performance by introducing quantum initialization to improve the initial population, integrating the slime mold algorithm for better exploration, and applying an average differential mutation strategy and nonlinear energy factor updates to strengthen both global and local search. Non-dominated sorting and crowding distance techniques are incorporated to enhance solution diversity and quality. The results show that compared to traditional HHO and HHO algorithms, the IHHO algorithm reduces average scheduling costs by 4.2% and 14.5%, scheduling time by 4.5% and 8.1%, and fuel consumption by 3.5% and 3.2%, respectively. This approach effectively reduces transfer path costs, saves energy, and improves operational efficiency, providing valuable insights for path planning and collaborative scheduling in multi-field harvesting and transportation in hilly areas.

A Fast Algorithm for the Real-Valued Combinatorial Pure Exploration of the Multi-Armed Bandit.

We study the real-valued combinatorial pure exploration problem in the stochastic multi-armed bandit (R-CPE-MAB). We study the case where the size of the action set is polynomial with respect to the number of arms. In such a case, the R-CPE-MAB can be seen as a special case of the so-called transductive linear bandits. We introduce the combinatorial gap-based exploration (CombGapE) algorithm, whose sample complexity upper-bound-matches the lower bound up to a problem-dependent constant factor. We numerically show that the CombGapE algorithm outperforms existing methods significantly in both synthetic and real-world data sets.

View: visual imitation learning with waypoints

Robots can use visual imitation learning (VIL) to learn manipulation tasks from video demonstrations. However, translating visual observations into actionable robot policies is challenging due to the high-dimensional nature of video data. This challenge is further exacerbated by the morphological differences between humans and robots, especially when the video demonstrations feature humans performing tasks. To address these problems we introduce Visual Imitation lEarning with Waypoints (VIEW), an algorithm that significantly enhances the sample efficiency of human-to-robot VIL. VIEW achieves this efficiency using a multi-pronged approach: extracting a condensed prior trajectory that captures the demonstrator’s intent, employing an agent-agnostic reward function for feedback on the robot’s actions, and utilizing an exploration algorithm that efficiently samples around waypoints in the extracted trajectory. VIEW also segments the human trajectory into grasp and task phases to further accelerate learning efficiency. Through comprehensive simulations and real-world experiments, VIEW demonstrates improved performance compared to current state-of-the-art VIL methods. VIEW enables robots to learn manipulation tasks involving multiple objects from arbitrarily long video demonstrations. Additionally, it can learn standard manipulation tasks such as pushing or moving objects from a single video demonstration in under 30 min, with fewer than 20 real-world rollouts. Code and videos here: https://collab.me.vt.edu/view/

Estimating Word Lengths for Fixed-Point DSP Implementations Using Polynomial Chaos Expansions

Efficient custom hardware motivates the use of fixed-point arithmetic in the implementation of digital signal-processing (DSP) algorithms. This conversion to finite precision arithmetic introduces quantization noise in the system, which affects the system’s performance. As a result, characterizing quantization noise and its effects within a DSP system is a challenge that must be addressed to avoid over-allocating hardware resources during implementation. Polynomial chaos expansion (PCE) is a method used to model uncertainty in engineering systems. Although it has been employed to analyze quantization effects in DSP systems, previous investigations have been limited in scope and scale. This paper introduces new techniques that allow the application of PCE to be scaled up to larger DSP blocks with many noise sources, as needed for building blocks in software-defined radios (SDRs). Design space exploration algorithms that leverage the accuracy of PCE to estimate bit widths for fixed-point implementations of DSP blocks in an SDR system are explored, and their advantages will be presented.

AI-driven automated discovery tools reveal diverse behavioral competencies of biological networks.

Many applications in biomedicine and synthetic bioengineering rely on understanding, mapping, predicting, and controlling the complex behavior of chemical and genetic networks. The emerging field of diverse intelligence investigates the problem-solving capacities of unconventional agents. However, few quantitative tools exist for exploring the competencies of non-conventional systems. Here, we view gene regulatory networks (GRNs) as agents navigating a problem space and develop automated tools to map the robust goal states GRNs can reach despite perturbations. Our contributions include: (1) Adapting curiosity-driven exploration algorithms from AI to discover the range of reachable goal states of GRNs, and (2) Proposing empirical tests inspired by behaviorist approaches to assess their navigation competencies. Our data shows that models inferred from biological data can reach a wide spectrum of steady states, exhibiting various competencies in physiological network dynamics without requiring structural changes in network properties or connectivity. We also explore the applicability of these 'behavioral catalogs' for comparing evolved competencies across biological networks, for designing drug interventions in biomedical contexts and synthetic gene networks for bioengineering. These tools and the emphasis on behavior-shaping open new paths for efficiently exploring the complex behavior of biological networks. For the interactive version of this paper, please visit https://developmentalsystems.org/curious-exploration-of-grn-competencies.

AI-driven automated discovery tools reveal diverse behavioral competencies of biological networks

Many applications in biomedicine and synthetic bioengineering rely on understanding, mapping, predicting, and controlling the complex behavior of chemical and genetic networks. The emerging field of diverse intelligence investigates the problem-solving capacities of unconventional agents. However, few quantitative tools exist for exploring the competencies of non-conventional systems. Here, we view gene regulatory networks (GRNs) as agents navigating a problem space and develop automated tools to map the robust goal states GRNs can reach despite perturbations. Our contributions include: (1) Adapting curiosity-driven exploration algorithms from AI to discover the range of reachable goal states of GRNs, and (2) Proposing empirical tests inspired by behaviorist approaches to assess their navigation competencies. Our data shows that models inferred from biological data can reach a wide spectrum of steady states, exhibiting various competencies in physiological network dynamics without requiring structural changes in network properties or connectivity. We also explore the applicability of these ‘behavioral catalogs’ for comparing evolved competencies across biological networks, for designing drug interventions in biomedical contexts and synthetic gene networks for bioengineering. These tools and the emphasis on behavior-shaping open new paths for efficiently exploring the complex behavior of biological networks. For the interactive version of this paper, please visit https://developmentalsystems.org/curious-exploration-of-grn-competencies.

Competing Bandits: The Perils of Exploration Under Competition

Most online platforms learn from interactions with users, and engage in exploration : making potentially suboptimal choices in order to acquire new information. We study the interplay between exploration and competition : how such platforms balance the exploration for learning and competition for users. We consider a stylized duopoly in which two firms face the same multi-armed bandit problem. Users arrive one by one and choose between the two firms, so that each firm makes progress on its bandit problem only if it is chosen. We study whether competition incentivizes the adoption of better algorithms. We find that stark competition disincentivizes exploration, leading to low welfare. However, weaker competition incentivizes better exploration algorithms and increases welfare. We investigate two channels for weakening the competition: stochastic user choice models and a first-mover advantage. Our findings speak to the competition-innovation relationship and the first-mover advantage in the digital economy.

Effective delta-labeling exploration algorithms for graph representation and DNA sequence alignment

Effective delta-labeling exploration algorithms for graph representation and DNA sequence alignment

Finite graph exploration by a mobile agent

The paper considers the task of finite connected graph exploration by a mobile agent. The mobile agent moves along the graph, reads and changes marks of the graph elements, and explores the graph based on this information. A new algorithm for finite undirected graph exploration of time complexity O(n^3), space complexity O(n^2) and the upper estimate of the number of transitions along the edges made by the agent O(n^3) is proposed. For the algorithm to work, the agent needs one color. The algorithm is based on depth-first traversal method.

Expert Experience Soft Actor–Critic for Unmanned Aerial Vehicle Dynamic Target Assignment

Deep reinforcement learning has proven highly effective in addressing sequential decision-making challenges, particularly in aerial combat involving unmanned aerial vehicles (UAVs). However, due to the prolonged decision-making process and uncertain opponent strategies in aerial combat, expert experience and manually crafted rules are often the preferred solutions. Consequently, the expert experience soft actor–critic (EESAC) algorithm is proposed to tackle the problem of dynamic target assignment for UAVs. EESAC features a customized reinforcement learning framework that includes a variable-scale reward mechanism to address the challenge of sparse rewards. Furthermore, a suppression exploration algorithm is implemented to reduce frequent target reassignments by UAVs during the initial stages, ensuring more stable and efficient mission execution. EESAC and comparator algorithms were integrated into an aerial combat simulation platform for a comprehensive evaluation. The training process and game results indicate that, compared to the comparator algorithms, EESAC significantly improves both exploration efficiency and win rate.

A Multi-Objective Optimization Model for Agricultural Crop Planting Strategies Using Enhanced Genetic Algorithms and Big Data Analysis

The purpose of this paper is to construct an optimization model of crop planting strategy based on improved genetic algorithm. In order to improve the efficiency of agricultural planting and maximize the economic benefits, this paper combines the linear programming model and the multi-matrix improved genetic algorithm (MI-GA), the improved NSGA-II algorithm of orthogonal experimental design, and the multiple nonlinear regression model based on heuristic exploration algorithm to explore the complex relationship between crop sales price, planting cost and yield per mu. Through the comprehensive analysis of planting constraints, market fluctuations and risk assessment of different crops, a more scientific and practical optimal planting scheme was proposed. Experimental results show that the constructed model has good convergence and optimization effect under the condition of multi-parameter change, which provides decision support for agricultural production and helps to realize the sustainable development of rural economy.

Sampling Efficient Deep Reinforcement Learning Through Preference-Guided Stochastic Exploration.

Stochastic exploration is the key to the success of the deep Q -network (DQN) algorithm. However, most existing stochastic exploration approaches either explore actions heuristically regardless of their Q values or couple the sampling with Q values, which inevitably introduce bias into the learning process. In this article, we propose a novel preference-guided ϵ -greedy exploration algorithm that can efficiently facilitate exploration for DQN without introducing additional bias. Specifically, we design a dual architecture consisting of two branches, one of which is a copy of DQN, namely, the Q branch. The other branch, which we call the preference branch, learns the action preference that the DQN implicitly follows. We theoretically prove that the policy improvement theorem holds for the preference-guided ϵ -greedy policy and experimentally show that the inferred action preference distribution aligns with the landscape of corresponding Q values. Intuitively, the preference-guided ϵ -greedy exploration motivates the DQN agent to take diverse actions, so that actions with larger Q values can be sampled more frequently, and those with smaller Q values still have a chance to be explored, thus encouraging the exploration. We comprehensively evaluate the proposed method by benchmarking it with well-known DQN variants in nine different environments. Extensive results confirm the superiority of our proposed method in terms of performance and convergence speed.

Analysis of the possibility of using exploration and learning algorithms in the production of castings

The research presented in the article indicates the process of building models based on machine learning algorithms, linear regression, decision trees, ensemble learning, random forest, ensemble averaging, Boosting, stacking, and support vector regression (SVR) algorithms. The basis for building these models are experimental data collected during research conducted at the Łukasiewicz Research Network-Krakow Institute of Technology. An analysis of the initial state and the analysis of the state of correlation in the set were performed, which facilitated the development of models. To increase the amount of data, augmentation was performed using the Bootstrapping. For selected results, castings were made and tested in real conditions. The research results indicate the possibility of identifying the most appropriate input parameters for specific production processes of austempered ductile iron (ADI), the possibility of predicting the expected mechanical parameters based on the indicated parameters of the production process, chemical composition, specific parameters of the heat treatment process, and the thickness of the target product. A set of such models constitutes the basis of the system, enabling the end user to estimate the final parameters of the casting planned to be produced.

Multi-Strategy Enhanced Parrot Optimizer: Global Optimization and Feature Selection.

Optimization algorithms are pivotal in addressing complex problems across diverse domains, including global optimization and feature selection (FS). In this paper, we introduce the Enhanced Crisscross Parrot Optimizer (ECPO), an improved version of the Parrot Optimizer (PO), designed to address these challenges effectively. The ECPO incorporates a sophisticated strategy selection mechanism that allows individuals to retain successful behaviors from prior iterations and shift to alternative strategies in case of update failures. Additionally, the integration of a crisscross (CC) mechanism promotes more effective information exchange among individuals, enhancing the algorithm's exploration capabilities. The proposed algorithm's performance is evaluated through extensive experiments on the CEC2017 benchmark functions, where it is compared with ten other conventional optimization algorithms. Results demonstrate that the ECPO consistently outperforms these algorithms across various fitness landscapes. Furthermore, a binary version of the ECPO is developed and applied to FS problems on ten real-world datasets, demonstrating its ability to achieve competitive error rates with reduced feature subsets. These findings suggest that the ECPO holds promise as an effective approach for both global optimization and feature selection.

User Preference Maps: Quantifying the Built Environment

The built environment in which we live holds the potential to provide life experiences that allow pedestrians to observe, feel, learn, and grow through their surroundings in everyday urban spaces. If a city offers opportunities for careful observation and exploration according to users’ preferences, it will become more appealing to many people. This study selected Midtown, New York, as the research site and collected a total of seven datasets based on 30 intersections in the area. The data, categorized into three main areas—activity, comfort, and natural elements—were evaluated, visualized, and restructured using a path exploration algorithm to produce a final user-based map. For this, 3D modeling software Rhino version 7, visual programming tool Grasshopper, and Grasshopper verion 2023 plugin programs were used. The final result included 3D route information, quantitative measurement data, and multidimensional visual materials. This approach presents an alternative to traditional route navigation based on uniform criteria and, through data-driven design, is believed to ultimately enhance walkability, activate urban spaces, and contribute to the development of sustainable cities. The scope of related research can further expand as the targets, duration, and methods of data collection continue to evolve and as case studies in various cities increase.

Explorative Binary Gray Wolf Optimizer with Quadratic Interpolation for Feature Selection.

The high dimensionality of large datasets can severely impact the data mining process. Therefore, feature selection becomes an essential preprocessing stage, aimed at reducing the dimensionality of the dataset by selecting the most informative features while improving classification accuracy. This paper proposes a novel binary Gray Wolf Optimization algorithm to address the feature selection problem in classification tasks. Firstly, the historical optimal position of the search agent helps explore more promising areas. Therefore, by linearly combining the best positions of the search agents, the algorithm's exploration capability is increased, thus enhancing its global development ability. Secondly, the novel quadratic interpolation technique, which integrates population diversity with local exploitation, helps improve both the diversity of the population and the convergence accuracy. Thirdly, chaotic perturbations (small random fluctuations) applied to the convergence factor during the exploration phase further help avoid premature convergence and promote exploration of the search space. Finally, a novel transfer function processes feature information differently at various stages, enabling the algorithm to search and optimize effectively in the binary space, thereby selecting the optimal feature subset. The proposed method employs a k-nearest neighbor classifier and evaluates performance through 10-fold cross-validation across 32 datasets. Experimental results, compared with other advanced algorithms, demonstrate the effectiveness of the proposed algorithm.

A novel autonomous exploration algorithm via LiDAR/IMU SLAM and hierarchical subsystem for mobile robot in unknown indoor environments

Abstract Autonomous exploration in unknown environments is an essential capability for mobile robots. The complexity of autonomous exploration, however, means that existing algorithms struggle to balance efficiency and comprehensiveness, causing low mapping accuracy and redundant path planning. To perform accurate and efficient exploration tasks, we have proposed a novel autonomous exploration algorithm via LiDAR/IMU (Inertial Measurement Unit) Simultaneous Localization and Mapping (SLAM) and hierarchical subsystem for mobile robot in unknown environments. Firstly, to enhance mapping accuracy for mobile robot exploration, LiDAR/IMU SLAM is improved with the assistance of backward propagation and iterated Kalman filter, and Bidirectional Rapidly–Exploring Random Trees* (BI–RRT*) is applied for efficient frontier point detection. Secondly, we optimize local path planning by leveraging information theory through perceptual quality evaluation, which is then integrated with global path planning utilizing an enhanced Travelling Salesman Problem solver and a sparse grid map to amplify exploration efficiency. Thirdly, an enhanced hierarchical autonomous exploration method for mobile robots is proposed, which incorporates local path planning for seamless navigation around highly promising exploration spots, coupled with global path planning to effectively interconnect various sub–regions. Finally, simulations and field tests have demonstrated that the proposed method explores an unknown indoor environment with a 30.8% reduction in exploration time and a 29.9% reduction in exploration path in comparison with Dynamic Stage Viewpoint Planner. The map constructed in this paper has more accurate details and exploration paths have been shortened to ensure effective exploration.

A Hybrid Prediction Model for Short-Term Load Forecasting in Power Systems

Short-term load forecasting (STLF) plays a vital role in effective power system management by assisting power dispatch centers in developing generation plans and ensuring smooth system operation. This study introduces a novel hybrid prediction model called iSSA-LSSVM to tackle the STLF challenge. By integrating the Salp Swarm Algorithm (SSA) with Least Squares Support Vector Machines (LSSVM), the iSSA-LSSVM model significantly improves LSSVM's prediction accuracy. One of the key contributions is the model's ability to autonomously ne-tune LSSVM hyperparameters, eliminating the need for manual adjustments and optimizing performance. Modifying the SSA within iSSA-LSSVM enhances the original algorithm's exploration and exploitation capabilities, ensuring better search efficiency and precision. Using a dataset with four independent variables as input and electrical power output as the target variable, the model demonstrates superior predictive performance. Comparative analysis with three other models shows that iSSA-LSSVM achieves a lower Mean Square Error (MSE) and faster convergence. This improvement in accuracy and efficiency enhances STLF, allowing power dispatch centers to develop more precise generation plans and ensure more reliable power system operation.