Multi-agent Exploration Research Articles

Multiple unmanned ship coverage and exploration in complex sea areas

This study addresses the complexities of maritime area information collection, particularly in challenging sea environments, by introducing a multi-agent control model for regional information gathering. Focusing on three key areas—regional coverage, collaborative exploration, and agent obstacle avoidance—we aim to establish a multi-unmanned ship coverage detection system. For regional coverage, a multi-objective optimization model considering effective area coverage and time efficiency is proposed, utilizing a heuristic simulated annealing algorithm for optimal allocation and path planning, achieving a 99.67% effective coverage rate in simulations. Collaborative exploration is tackled through a comprehensive optimization model, solved using an improved greedy strategy, resulting in a 100% static target detection and correct detection index. Agent obstacle avoidance is enhanced by a collision avoidance model and a distributed underlying collision avoidance algorithm, ensuring autonomous obstacle avoidance without communication or scheduling. Simulations confirm zero collaborative failures. This research offers practical solutions for multi-agent exploration and coverage in unknown sea areas, balancing workload and time efficiency while considering ship dynamics constraints.

Imagine, Initialize, and Explore: An Effective Exploration Method in Multi-Agent Reinforcement Learning

Effective exploration is crucial to discovering optimal strategies for multi-agent reinforcement learning (MARL) in complex coordination tasks. Existing methods mainly utilize intrinsic rewards to enable committed exploration or use role-based learning for decomposing joint action spaces instead of directly conducting a collective search in the entire action-observation space. However, they often face challenges obtaining specific joint action sequences to reach successful states in long-horizon tasks. To address this limitation, we propose Imagine, Initialize, and Explore (IIE), a novel method that offers a promising solution for efficient multi-agent exploration in complex scenarios. IIE employs a transformer model to imagine how the agents reach a critical state that can influence each other's transition functions. Then, we initialize the environment at this state using a simulator before the exploration phase. We formulate the imagination as a sequence modeling problem, where the states, observations, prompts, actions, and rewards are predicted autoregressively. The prompt consists of timestep-to-go, return-to-go, influence value, and one-shot demonstration, specifying the desired state and trajectory as well as guiding the action generation. By initializing agents at the critical states, IIE significantly increases the likelihood of discovering potentially important under-explored regions. Despite its simplicity, empirical results demonstrate that our method outperforms multi-agent exploration baselines on the StarCraft Multi-Agent Challenge (SMAC) and SMACv2 environments. Particularly, IIE shows improved performance in the sparse-reward SMAC tasks and produces more effective curricula over the initialized states than other generative methods, such as CVAE-GAN and diffusion models.

Multi-Agent Exploration With Similarity Score Map and Topological Memory

Multi-Agent Exploration With Similarity Score Map and Topological Memory

Active Neural Topological Mapping for Multi-Agent Exploration

Active Neural Topological Mapping for Multi-Agent Exploration

MAES: a ROS 2-compatible simulation tool for exploration and coverage algorithms

With the aim of allowing the efficient and realistic simulation of swarm algorithms for exploration and coverage, we present the tool Multi-Agent Exploration Simulator (MAES), which is an open-source physics-based discrete step multi-robot simulator. MAES features movement in a continuous 2D space, realistic physics based on the Unity framework, advanced visualization techniques such as heatmaps, custom wireless signal degradation, both randomly generated and custom user-provided maps, and a ROS (Robot Operating System) interface. This latter characteristic could allow to port the simulated algorithms to real-world robots. We present performance tests, conducted with rather modest hardware, showing that MAES is able to simulate up to 5 robots in ROSMode (using the ROS integration) and up to 120 robots in UnityMode (development performed directly into the C# Unity Editor). A usability test was conducted which hinted that the target audience of robotics researchers and developers is able to quickly install, setup, and use MAES for implementing simple robot logic.

Multi-Agent Variational Approach for Robotics: A Bio-Inspired Perspective.

This study proposes an adaptable, bio-inspired optimization algorithm for Multi-Agent Space Exploration. The recommended approach combines a parameterized Aquila Optimizer, a bio-inspired technology, with deterministic Multi-Agent Exploration. Stochastic factors are integrated into the Aquila Optimizer to enhance the algorithm's efficiency. The architecture, called the Multi-Agent Exploration-Parameterized Aquila Optimizer (MAE-PAO), starts by using deterministic MAE to assess the cost and utility values of nearby cells encircling the agents. A parameterized Aquila Optimizer is then used to further increase the exploration pace. The effectiveness of the proposed MAE-PAO methodology is verified through extended simulations in various environmental conditions. The algorithm viability is further evaluated by comparing the results with those of the contemporary CME-Aquila Optimizer (CME-AO) and the Whale Optimizer. The comparison adequately considers various performance parameters, such as the percentage of the map explored, the number of unsuccessful runs, and the time needed to explore the map. The comparisons are performed on numerous maps simulating different scenarios. A detailed statistical analysis is performed to check the efficacy of the algorithm. We conclude that the proposed algorithm's average rate of exploration does not deviate much compared to contemporary algorithms. The same idea is checked for exploration time. Thus, we conclude that the results obtained for the proposed MAE-PAO algorithm provide significant advantages in terms of enhanced map exploration with lower execution times and nearly no failed runs.

MUI-TARE: Cooperative Multi-Agent Exploration With Unknown Initial Position

MUI-TARE: Cooperative Multi-Agent Exploration With Unknown Initial Position

Real-Time Hierarchical Map Segmentation for Coordinating Multirobot Exploration

Coordinating a team of autonomous agents to explore an environment can be done by partitioning the map of the environment into segments and allocating the segments as targets for the individual agents to visit. However, given an unknown environment, map segmentation must be conducted in a continuous and incremental manner. In this paper, we propose a novel real-time hierarchical map segmentation method for supporting multi-agent exploration of indoor environments, wherein clusters of regions of segments are formed hierarchically from randomly sampled points in the environment. Each cluster is then assigned with a cost-utility value based on the minimum cost possible for the agents to visit. In this way, map segmentation and target allocation can be performed continually in real-time to efficiently explore the environment. To evaluate our proposed model, we conduct extensive experiments on map segmentation and multi-agent exploration. The results show that the proposed method can produce more accurate and meaningful segments leading to a higher level of efficiency in exploring the environment. Furthermore, the robustness tests by adding noises to the environments were conducted to simulate the performance of our model in the real-world environment. The results demonstrate the robustness of our model in map segmentation and multi-agent environment exploration.

The StarCraft Multi-Agent Exploration Challenges: Learning Multi-Stage Tasks and Environmental Factors Without Precise Reward Functions

In this paper, we propose a novel benchmark called the StarCraft Multi-Agent Exploration Challenges(SMAC-Exp), where agents learn to perform multi-stage tasks and to use environmental factors without precise reward functions. The previous challenges (SMAC) recognized as a standard benchmark of Multi-Agent Reinforcement Learning are mainly concerned with ensuring that all agents cooperatively eliminate approaching adversaries only through fine manipulation with obvious reward functions. SMAC-Exp, on the other hand, is interested in the exploration capability of MARL algorithms to efficiently learn implicit multi-stage tasks and environmental factors as well as micro-control. This study covers both offensive and defensive scenarios. In the offensive scenarios, agents must learn to first find opponents and then eliminate them. The defensive scenarios require agents to use topographic features. For example, agents need to position themselves behind protective structures to make it harder for enemies to attack. We investigate a total twelve MARL algorithms under both sequential and parallel episode settings of SMAC-Exp and observe that recent approaches perform well in similar settings to the previous challenge, but we discover that current multi-agent approaches place relatively less emphasis on exploration perspectives. To a limited extent, we observe that an enhanced exploration method has a positive effect on SMAC-Exp, however there is still gap that state-of-art algorithms cannot resolve the most challenging scenarios of SMAC-Exp. Consequently, we propose a new axis for future research into Multi-Agent Reinforcement Learning studies.

MR-TopoMap: Multi-Robot Exploration Based on Topological Map in Communication Restricted Environment

Multi-robot exploration in unknown environments is a fundamental task for a multi-robot system, involving inter-robot communication through messages among the robots. However, in a restricted communication environment, the limited communication resources become the system's bottleneck due to a large amount of data in the occupancy grid map. Hence, to enhance multi-agent exploration in communication-constrained environments, this letter develops a method to build topological maps while the robot moves in the environment and an exploration strategy based on the created topological map. The latter map comprises a set of vertices and edges connecting the vertices, where each vertex represents a specific area embedded with a descriptor extracted by visually observing this area and recognizing it utilizing descriptors. Each robot has its local grid map stored for path planning, not shared between them. Considering the exploration task, a robot's ability to choose a proper direction depends on the other robot's locations and the unexplored areas. Our exploration framework is evaluated on the Gazebo simulator and real robots, increasing the exploration efficiency by 23% <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula> 77%. Compared with the occupancy grid map scheme, our method's data transfer is reduced by 84% <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula> 90%.

Indoor Visual Exploration with Multi-Rotor Aerial Robotic Vehicles.

In this work, we develop a reactive algorithm for autonomous exploration of indoor, unknown environments for multiple autonomous multi-rotor robots. The novelty of our approach rests on a two-level control architecture comprised of an Artificial-Harmonic Potential Field (AHPF) for navigation and a low-level tracking controller. Owing to the AHPF properties, the field is provably safe while guaranteeing workspace exploration. At the same time, the low-level controller ensures safe tracking of the field through velocity commands to the drone’s attitude controller, which handles the challenging non-linear dynamics. This architecture leads to a robust framework for autonomous exploration, which is extended to a multi-agent approach for collaborative navigation. The integration of approximate techniques for AHPF acquisition further improves the computational complexity of the proposed solution. The control scheme and the technical results are validated through high-fidelity simulations, where all aspects, from sensing and dynamics to control, are incorporated, demonstrating the capacity of our method in successfully tackling the multi-agent exploration task.

A Centralized Strategy for Multi-Agent Exploration

This paper introduces recently developed Aquila Optimization Algorithm specifically configured for Multi-Robot space exploration. The proposed hybrid framework "Coordinated Multi-Robot Exploration Aquila Optimizer" (CME-AO) is a unique combination of both deterministic Coordinated Multi-robot Exploration (CME) and a swarm based methodology, known as Aquila Optimizer (AO). A novel parallel communication protocol is also embedded to improve multi-robot space exploration process while simultaneously minimizing both the computation complexity and time. This ensures acquisition of a optimal collision-free path in a barrier-filled environment via generating a finite map. The architecture starts by determining the cost and utility values of neighbouring cells around the robot using deterministic CME. Aquila Optimization technique is then incorporated to increase the overall solution accuracy. Numerous simulations under different environmental conditions were then performed to validate the effectiveness of the proposed algorithm. Algorithm consistency aspects in achieving the expected results (area explored rate and time) is demonstrated through statistical means. A perspective analysis is then performed by comparing the performance of the CME-AO algorithm with latest state of art contemporary algorithms namely conventional CME and CME-WO (CME Whale Optimizer). The comparison duly accommodates all pertinent aspects such as % area explored, number of failed runs, and time taken for map exploration for different environments. Results indicate that the proposed algorithm presents two distinct advantages over the other conventional state of the art CME based techniques a) enhanced map exploration in cluttered environment and b) significantly reduced computation complexity and execution time, with almost no fail runs. This makes the suggested methodology particularly suitable for on-board utilization in an obstacle-cluttered environment, where other techniques either fails (stuck locally) or takes longer exploration time.

Density-aware decentralised multi-agent exploration with energy constraint based on optimal transport theory

This paper addresses a density-aware multi-agent exploration problem based on an Optimal Transport (OT) theory while considering energy constraints of a multi-agent system. The density-aware exploration means how a team of agents (robots) cover a given domain, reflecting a priority of areas of interest represented by a density distribution, rather than simply following a preset of uniform patterns. To achieve the density-aware multi-agent exploration, the optimal transport theory that quantifies a distance between two density distributions is employed as a tool, which also serves as a means of similarity measure. Energy constraints for a multi-agent system are then incorporated into the OT-based density-aware multi-agent exploration scheme. The proposed method is developed targeting a decentralised control to cope with more realistic scenarios such as communication range limits between agents. To measure the exploration efficiency, the upper bound of the similarity measure is proposed, which is computationally tractable. The developed multi-agent exploration scheme is applicable to a time-varying distribution as well, where a spatio-temporal evolution of the given reference distribution is desired. To validate the proposed method, multiple simulation results are provided.

Multi-agent exploration of spatial dynamical processes under sparsity constraints

This paper addresses the development of an efficient information gathering and exploration strategy for robotic missions when a high level of autonomy is expected. A multi-agent system is considered, which consists of several mobile sensing platforms with the goal to identify the parameters of a spatio-temporal process modeled by a partial differential equation. Specifically, an exploration of a diffusion process driven by an unknown number of sparsely located sources is considered. A probabilistic approach toward partial differential equations and sparsity constraints modeling with factor graphs is developed and realized by a customized message passing algorithm. The algorithm permits efficient identification of source parameters: the number of sources, their locations and amplitudes. In addition, an exploration strategy to guide the agents to more informative sampling locations is proposed; this accelerates identification of the source parameters. The message passing implementation facilitates efficient distributed implementation, which is of significant advantage with respect to scalability, computational complexity and an implementation in a multi-agent system. The effectiveness of the algorithm is demonstrated using synthetic data in simulations.

Bayesian Online Learning for Information-based Multi-Agent Exploration with Unknown Radio Signal Distribution

Exploring an unknown environment with multiple robots is an enabling technology for many useful applications. This paper investigates decentralized motion planning for multi-agent exploration in a field with unknown received signal strength (RSS) distribution. The environment is modelled with a Gaussian process using Bayesian online learning by sharing the information obtained from the measurement history of each robot. Then we use the mean function of the Gaussian process to infer the multiple RSS source locations. The inferred source locations are modelled as the probability distribution using Gaussian mixture-probability hypothesis density (GM-PHD) filter. This modelling enables nonparametric approximation of mutual information between source locations and future robot positions. We combine the variance function of the Gaussian process and the mutual information to design an informative and noise-robust planning algorithm for multiple robots. The experimental performance is analyzed by comparing with the variance-based planning algorithm.

Utilizing costly coordination in multi-agent joint exploration1

This paper studies distributed cooperative multi-agent exploration methods in settings where the exploration is costly and the overall performance measure is determined by the minimum performance achieved by any of the individual agents. Such an expl

Information Sharing Under Costly Communication in Joint Exploration

This paper studies distributed cooperative multi-agent exploration methods in settings where the exploration is costly and the overall performance measure is determined by the minimum performance achieved by any of the individual agents. Such an exploration setting is applicable to various multi-agent systems, e.g., in Dynamic Spectrum Access exploration. The goal in such problems is to optimize the process as a whole, considering the tradeoffs between the quality of the solution obtained and the cost associated with the exploration and coordination between the agents. Through the analysis of the two extreme cases where coordination is completely free and when entirely disabled, we manage to extract the solution for the general case where coordination is taken to be costly, modeled as a fee that needs to be paid for each additional coordinated agent. The strategy structure for the general case is shown to be threshold-based, and the thresholds which are analytically derived in this paper can be calculated offline, resulting in a very low online computational load.

Sequential multi-agent exploration for a common goal

Motivated by applications in Dynamic Spectrum Access Networks, we focus on a system in which a few agents are engaged in a costly individual exploration process where each agent's benefit is determined according to the minimum obtained value. Such an exploration pattern is applicable to many systems, including shipment and travel planning. This paper formally introduces and analyzes a sequential variant of the general model. According to that variant, only a single agent engages in exploration at any given time, and when an agent initiates its exploration, it has complete information about the minimum value obtained by the other agents so far. We prove that the exploration strategy of each agent, according to the equilibrium of the resulting Stackelberg game, is reservation-value based, and show how the reservation values can be calculated. We also analyze the agents' expected-benefit maximizing exploration strategies when they are fully cooperative i.e., when they aim to maximize the expected joint benefit. The equilibrium strategies and the expected benefit of each agent are illustrated using a synthetic homogeneous environment, thereby demonstrating the properties of this new exploration scheme and the benefits of cooperation.

Practical Issues in Deploying Mobile Agents to Explore a Sensor-Instrumented Environment

When an emergency occurs within a building, it is safer to send autonomous mobile agents instead of human responders, to explore the area and identify hazards and victims. Existing exploration algorithms (Svennebring, J. and Koenig, S. (2004) Building terrain-covering Ant robots: a feasibility study. Auton. Robots, 16, 313–332, Ferranti, E., Trigoni, N. and Levene, M. (2007) Brick&Mortar: An On-Line Multi-Agent Exploration Algorithm. ICRA07: Proc. 2007 IEEE Int. Conf. Robotics and Automation, April, pp. 761–767. IEEE Press) allow mobile agents to make distributed navigation decisions by communicating with nearby fixed sensors embedded in the environment. These algorithms are very efficient in terms of exploration time, but they have only been evaluated in simulation environments, where idealized assumptions were made regarding the ability of agents to localize sensors and move accurately towards them. The objective of this work is to investigate practical issues of building a real testbed of mobile agents and fixed sensors, and implementing exploration algorithms in such a testbed. In particular, we describe our experiences from building a real system consisting of a Surveyor SRV-1 robot and Tmote Sky sensors running the Contiki OS (Dunkels, A., Gronvall, B. and Voigt, T. (2004) Contiki — A Lightweight and Flexible Operating System for Tiny Networked Sensors. Proc. 1st Annual IEEE Int. Workshop on Embedded Networked Sensors, November, pp. 455–462). We select two existing exploration algorithms, Ants (Svennebring, J. and Koenig, S. (2004) Building terrain-covering Ant robots: a feasibility study. Auton. Robots, 16, 313–332) and Brick&Mortar (Ferranti, E., Trigoni, N. and Levene, M. (2007) Brick&Mortar: An On-Line Multi-Agent Exploration Algorithm. ICRA07: Proc. 2007 IEEE Int. Conf. Robotics and Automation, April, pp. 761–767. IEEE Press), and discuss challenges in trying to implement them in our testbed. To address these challenges, we propose practical solutions that allow a mobile agent to: (i) identify and localize fixed sensors deployed in its vicinity and (ii) accurately move towards a carefully selected fixed sensor. Using our real network deployment, we derive realistic models of localization and odometry errors. We then insert these error models into a realistic simulation environment, in order to extensively compare Ants and Brick&Mortar, and measure their performance degradation as a result of introducing realistic errors.

Investigating the Adaptiveness of Communication in Multi-Agent Behavior Coordination

Some previous studies of the adaptiveness of communication for coordination have found communication beneficial, others have not. We claim that this results from the lack of a systematic examination of important variables such as communication range, sensory range, and environmental conditions. We present an extensive series of simulations exploring how these parameters effect the utility of communication for coordination in the multi-agent territory exploration (MATE( n)) task. MATE( n) requires agents to visit all checkpoints in the environment in as little time as possible; n agents must be at a checkpoint simultaneously for it to be counted “visited.” A comparison of the absolute performance of communicating and non-communicating agents on MATE( n) (i.e., performance without regard to cost) finds that communication can be beneficial. A subsequent analysis of the results establishes constraints on the cost of communication for it to provide relative performance benefit (i.e., absolute performance scaled by cost).