Swarm Control Research Articles

Application of Deep Reinforcement Learning to UAV Swarming for Ground Surveillance.

This paper summarizes in depth the state of the art of aerial swarms, covering both classical and new reinforcement-learning-based approaches for their management. Then, it proposes a hybrid AI system, integrating deep reinforcement learning in a multi-agent centralized swarm architecture. The proposed system is tailored to perform surveillance of a specific area, searching and tracking ground targets, for security and law enforcement applications. The swarm is governed by a central swarm controller responsible for distributing different search and tracking tasks among the cooperating UAVs. Each UAV agent is then controlled by a collection of cooperative sub-agents, whose behaviors have been trained using different deep reinforcement learning models, tailored for the different task types proposed by the swarm controller. More specifically, proximal policy optimization (PPO) algorithms were used to train the agents' behavior. In addition, several metrics to assess the performance of the swarm in this application were defined. The results obtained through simulation show that our system searches the operation area effectively, acquires the targets in a reasonable time, and is capable of tracking them continuously and consistently.

Alternative locust control with a dilutable linseed oil emulsion

AbstractLocust swarm control is still a main topic in agriculture and is often based on the extensive use of chemical agents that can be problematic for the environment and human health. Currently, only a few alternative control agents are available on the market, such as neem oil, the fungus Metarhizium sp., and the microsporidian Paranosema locustae. In this study, we improved the formulation of a botanical insecticide that is based on linseed oil and was found to be effective against the desert locust, Schistocerca gregaria. To make sure that linseed oil is most effective, we first tested several natural plant oils in a comparative laboratory study. Linseed oil performed best and resulted in 100% mortality within only 2 days after a single spray treatment. However, this linseed oil formulation contains three expensive essential oils, thus we further improved it to avoid costly essential oils. A much lower concentration of linseed oil was also found to be highly effective and killed adult S. gregaria within 3 days in an experiment performed under semifield conditions. Furthermore, a concentrated stock solution of this botanical insecticide was developed to increase stability and reduce shipping costs in areas where water for dilution is available. Mechanical treatment, applied during the preparation of this linseed oil emulsion, resulted in the formation of micro‐colloids, which prevent the separation of phases and enable 1:16 dilution with water containing bicarbonate. Interestingly, only 24 h after single spray treatment, locusts became immobile and most of them were unable to jump, which would make them easy prey for natural enemies. Later, insects died due to the hardening of the linseed oil emulsion. This novel linseed oil emulsion is harmless for humans, rather easy to produce, and remains stable over weeks in its concentrated form, which suggests that it is promising alternative agent for locust control.

Two‐order cooperative optimization of swarm control based on reinforcement learning

AbstractThis paper presents a study of the cooperative optimal swarm control problem for two‐order multi‐agent systems with partially unknown nonlinear functions. Unlike traditional approaches that consider a single error, this paper proposes to use multi‐order errors in the performance index function to achieve optimal control performance. Additionally, different proportional coefficients are assigned to illustrate the varying influences of each sequence error, and a two‐order cooperative (TOC)performance index function is designed. To address the influence of unknown nonlinear functions, a swarm control system based on sliding mode control with an actor‐critic network is constructed, which increases the applicability of the proposed method to a variety of dynamic models. Furthermore, to alleviate the computational pressure caused by the multi‐order errors in the TOC performance index function, a new reinforcement learning (RL)‐based sliding mode swarm controller is designed. The stability of the proposed controller is demonstrated using the Lyapunov function. Finally, the control model and control rate are applied to a quadrotor unmanned aerial vehicle system, and simulation results demonstrate that the multi‐agent systems can effectively achieve swarm control.Impact Statement: This paper proposes a reinforcement learning‐based sliding mode control strategy for the cooperative optimal swarm control problem, where the nonlinear functions of two‐order multi‐agent systems are only partially known. In addition, we also propose a cooperative performance index function, which takes into account multi‐order errors for optimizing the performance. This contribution is significant for research in sliding mode control strategies and error co‐optimization.

Neural Swarm Control Algorithm for Underwater Vehicles

The paper presents the application of an evolutionary recurrent neural network to control the swarm of underwater vehicles. In the swarm, one vehicle is the leader and the others are followers. The leader leads the swarm along a predefined trajectory without regard for the followers while the followers follow the leader and avoid collisions with all other vehicles. Avoiding collisions by the swarm with external obstacles is done by changing the depth. The leader is responsible for detecting the obstacles and informing all the followers about the need to change the depth. To follow the leader, the followers use the information about the distance to it. Directional information is unavailable to them. To avoid collisions inside the swarm, the followers use shortrange sensors.

Dynamic Path Planning and Motion Control of Microrobotic Swarms for Mobile Target Tracking

Magnetic field-driven microrobotic swarms have drawn extensive attention, especially in the field of automatic control. Realizing dynamic path planning and motion control of microrobotic swarms for mobile target tracking is one of the important tasks that still remains unsolved. In this paper, we firstly present an enhanced bidirectional rapidly-exploring random tree star (EB-RRT*) algorithm considering the physical size of the swarm to dynamically plan the optimal path for obstacle avoidance. An image-guided motion controller, which consists of a direction controller and a Genetic Algorithm based Linear Quadratic Regulator (GA-LQR) velocity controller, is then proposed to realize mobile target tracking using microrobotic swarms. Targeted bursting algorithm is subsequently developed to meet the requirement of tracking high-speed ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , 20 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mu m/s$</tex-math> </inline-formula> ) mobile targets. Simulations are performed to validate the proposed methods and obtain the proper ranges of the input parameters for the controllers. Finally, the control effectiveness of mobile target tracking in different conditions and environments is validated by experimental results. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The motivation of this work is to develop an effective control scheme for mobile target tracking using microrobotic swarms. Conventional control schemes mainly focus on the control of single microrobots to reach static targets, and thus the desired path is fixed once planned. In addition, the motion of single monolithic microrobots can be modelled precisely. However, in mobile target tracking using microrobotic swarms, dynamic planning algorithms are demanded to frequently update the desired path. Swarms consisting of millions of micro-agents are also difficult to be modelled due to the complex agent-agent interactions. In this work, an effective control scheme consisting of a dynamic path planner, a motion controller and a targeted bursting unit is developed. Real-time dynamic paths will be planned even though the positions of the swarm and the target change rapidly. The precise control of the swarm direction and velocity are achieved, and moreover, using the targeted bursting algorithm, the swarm can be accelerated to approach mobile targets accurately with higher efficiency. Experimental results validates the proposed tracking strategy in different environments with virtual obstacles. The proposed control scheme paves the way for a better understanding of advanced motion control methods for microrobotic swarms.

Novel Coil Array Design and Modeling for Independent Control of Multiple Magnetic Microrobots

The independent control of microrobot swarms is a major challenge in the field of micro/nano manipulation. In this paper, a new localized magnetic field generating system is presented that can be used for the independent control of multiple magnetic microrobots. Such a system will allow for cooperative or parallel microassembly tasks with teams of magnetic microrobots. Firstly, the performance of the novel circular microcoil is modeled by analyzing the effect of input current to study the magnetic field distribution characteristics in the workspace, which is validated by finite element analysis and simulation; then, an actuation and control strategy of microrobots for microcoil array is proposed. Meanwhile, the independent closed-loop control of the microrobot is realized based on visual feedback; finally, an experimental platform for the microcoil actuation system is built to validate the newly designed structure. The results show that the microrobot can move flexibly and accurately with the tracking error within 0.444mm. At the same time, the system is capable of independently controlling multiple microrobots and holds great potential in performing micromanipulation tasks.

Differential flatness-based distributed control of underactuated robot swarms

Differential flatness-based distributed control of underactuated robot swarms

Comparative Analysis of Different UAV Swarm Control Methods on Unmanned Farms

Unmanned farms employ a variety of sensors, automated systems, and data analysis techniques to enable fully automated and intelligent management. This not only heightens agricultural production efficiency but also reduces the costs associated with human resources. As integral components of unmanned farms’ automation systems, agricultural UAVs have been widely adopted across various operational stages due to their precision, high efficiency, environmental sustainability, and simplicity of operation. However, present-day technological advancement levels and relevant policy regulations pose significant restrictions on UAVs in terms of payload and endurance, leading to diminished task efficiency when a single UAV is deployed over large areas. Accordingly, this paper aggregates and analyzes research pertaining to UAV swarms from databases such as Google Scholar, ScienceDirect, Scopus, IEEE Xplorer, and Wiley over the past decade. An initial overview presents the current control methods for UAV swarms, incorporating a summary and analysis of the features, merits, and drawbacks of diverse control techniques. Subsequently, drawing from the four main stages of agricultural production (cultivation, planting, management, and harvesting), we evaluate the application of UAV swarms in each stage and provide an overview of the most advanced UAV swarm technologies utilized therein. Finally, we scrutinize and analyze the challenges and concerns associated with UAV swarm applications on unmanned farms and provide forward-looking insights into the future developmental trajectory of UAV swarm technology in unmanned farming, with the objective of bolstering swarm performance, scalability, and adoption rates in such settings.

Studying the Performance Features of the Scenario Algorithm in the Control System of an Unmanned Aerial System

In this paper, we consider the results of a full-scale experiment carried out under conditions most closely resembling real operating ones. The goal of the experiment is to identify the performance features of the control system inside an unmanned aerial system (UAS) when using the swarm control method of a decentralized UAS control strategy. The considered flight performance characteristics of a single UAV provide an opportunity to analyze the operation feasibility of the presented UAS to solve other kinds of problems. The results of the presented experiment enable the development of a set of recommendations for further research directions. Also, they provide future UAV operators with better and faster achievement of practical goals during the practical operation of UAS in various conditions. The described features and disadvantages of various BAS control strategies help determine probable options for the BAS configuration depending on the nature of the tasks to be solved later.

Leader-Follower Based Formation Control of Heterogeneous UAV-AGV Multi-Agent System

This paper deals with a leader-follower formation control of a heterogenous robot swarm. The heterogeneous swarm consists of unmanned ground vehicles (UGV) and unmanned aerial vehicles (UAV). The ground robot is the leader robot, and the drones are the followers. A centralized system receives the information about the robots, and the decision for the robots is made from there. The robots create the V formation shape and are assigned to the formation points using the Hungarian algorithm. The robots go to the formation points with a proportional controller. The system was developed using the ROS2 framework. Turtlebot3 and Crazyflie robots were used for the robot swarm. The study was tested in Webots simulation environment. Different tests were performed, and the results were observed.

On the anatomy and structural control of a dyke swarm that fed caldera-forming ignimbrite eruptions

The evolution of eruptive vents related to calderas is not fully understood. We focus on a structural, rock-magnetic and geochemical investigation of a 314 Ma rhyolite dyke swarm associated with the late-orogenic Altenberg–Teplice Caldera, Bohemian Massif, eastern Variscan belt. The whole-rock major element, trace element and Nd–Pb isotope geochemistry along with the published U–Pb zircon geochronology link the extra-caldera dyke swarm with intra-caldera ignimbrites. The magnetic fabrics determined using the anisotropy of magnetic susceptibility are interpreted to record a continuum from magma ascent to emplacement and eruption during sinistral shearing. The latter evidences an interplay with regional tectonics associated with the activity of crustal-scale shear zones. The sinistral kinematics and strike of the dyke swarm, the elongation of caldera intrusive units, and the kinematics of major caldera faults are consistent with the dextral Riedel shear system, where the dykes correspond to antithetic R’ or X shears. Such a kinematic configuration implies that the maximum and minimum principal stresses were oriented roughly north–south and east–west, respectively. The relation between the stress field and the caldera elongation and orientation is not typical. We suggest that a pre-existing mutually perpendicular set of cross-cutting structural lineaments largely controlled the magma chamber and caldera formation. Supplementary material : The whole-rock major, trace element and isotope geochemical tables, magnetic fabrics source data and details of methods are available at https://doi.org/10.6084/m9.figshare.c.6715893

MBEANN for Robotic Swarm Controller Design and the Behavior Analysis for Cooperative Transport

This study focuses on mutation-based evolving artificial neural network (MBEANN), a topology and weight evolving artificial neural network (TWEANN) algorithm. TWEANN optimizes both the connection weights and neural network structure. Primarily, MBEANN uses only mutations to evolve artificial neural networks. An individual in an MBEANN is designed to have a set of sub-networks called operons. Operons are expected to have functions during evolution because they do not recombine with other operons. In this study, we applied MBEANN to design a controller for a robotic swarm on cooperative transport, where the following canonical evolving artificial neural network (EANN) methods do not work well. For comparison with MBEANN, we used an EANN with a fixed network structure and neuroevolution of augmenting topologies (NEAT), which is a widely used TWEANN algorithm. We confirmed that the robot controller that evolved with the MBEANN outperformed the structure-fixed EANN and NEAT controllers. In addition, we investigated the behavior of the swarm robot obtained using the proposed method, in which we deactivated each operon to extract its function. The results show that operons could have their functions, and that several operons could strengthen one another’s functions.

Adapting constrained scales to observation resolution in ocean forecasts

To advance predictive skill, ocean forecast systems must exploit local high resolution observations. The recent Sub-Mesoscale Ocean Dynamics Experiment (S-MODE) provides an opportunity to examine the data assimilation issues that can limit predictive skill. A unique swarm control system guided nine ocean gliders accounting for all satellite and in situ data to provide high resolution observations for three ocean forecast experiments: 1) no glider data assimilated, 2) assimilation with glider data through a previous assimilation approach, and 3) adaptive assimilation of the glider data in which smaller scales are corrected in the vicinity of the local high resolution observations. The assimilation adaptation used a spatially varying horizontal decorrelation scale enabling corrections at smaller scales where high resolution in situ data are concentrated. When measured by spatial scales along the glider paths, skill in temperature advances from scales larger than about 220 km wavelength when utilizing the glider data with prior assimilation down to scales larger than 80 km wavelength with the adaptive assimilation. Skill in salinity advances from scales of approximately 300 km to scales of 200 km, and skill in steric height advances from 140 km to 63 km. Additionally, conductivity, temperature, and depth observations from an EcoCTD instrument provide independent data for confirmation. The results imply that as ocean observing systems advance, ocean forecast systems must adapt to use local high resolution observations.

Automatic Navigation of Microswarms for Dynamic Obstacle Avoidance

Control and navigation of microrobotic swarms have drawn extensive attention recently. Avoiding dynamic obstacles using swarms is one of the major challenges that still remain unsolved. In this work, we develop a control strategy to navigate microrobotic swarms to targeted positions while avoiding dynamic obstacles. We first propose a criterion to evaluate the real-time locomotion efficiency during dynamic obstacle avoidance, i.e., the swarm moving direction and the distance between the swarm and the target. Subsequently, a hierarchical radar with three functional boundaries (detection, safety, and prediction circle) is designed for swarms. The optimal moving direction of the swarm with the existence of dynamic obstacles is selected based on the three circles. The effectiveness of the algorithm is validated by simulations and experiments. Using the proposed strategy, the swarm is capable of avoiding multiple moving obstacles and reaching the predefined target. Finally, to show the compatibility of the proposed control method, the swarm is deployed in a micromaze with different dynamic obstacles, and the results also validate the effectiveness of the strategy.

Influence-Based Nano Fuzzy Swarm Oxygen Deficiency Detection and Therapy

Oxygen deficiency is a serious health problem that may occur as a result of many diseases. In this article, we present an influence-based nano fuzzy swarm (INFS) for oxygen deficiency detection and therapy using a swarm of oxygen carrier nanomachines operating in three cognitive fields of control, influence, and interest. In particular, we propose a long short-term memory (LSTM) deep neural network for detecting apnea by analyzing the irregular peripheral oxygen saturation (SpO2) signal. Using the proposed sleep-in-the-loop strategy and the desaturated blood biomarkers, including oxygen and hydrogen ion concentrations, an in-silico multithreshold nano fuzzy swarm noninvasive therapeutic method is then performed. We also analytically prove the stability of the INFS using swarm control theory. We apply our strategy to sleep apnea, as one of the most common instances of oxygen deficiency. Furthermore, we compare the accuracy of INSF by using LSTM, bidirectional LSTM (BiLSTM), multilayer perceptron (MLP), convolutional neural network (CNN), and support vector machines (SVM). The detection and therapy results are then compared with other apnea detection methods. The input variables and structure of INSF, i.e., the number of rules and width of membership functions, are studied in terms of robustness to noise. As the results show, the proposed artificial intelligence (AI)-based noninvasive nano detection and therapy method could outperform the competing approaches in treating oxygen deficiency emergencies such as apnea.

Swarm control based on artificial potential field method with predicted state and input threshold

In this paper, an improved artificial potential field (APF) method with predictive state is proposed to reduce the influence of communication delay and physical delay on swarm control. The input threshold is introduced to optimize the problem of excessive APF method control input. The control of large-scale swarm systems is challenging due to the complexity of the communication topology. The second-order communication topology is introduced to simplify the coupling relationship of the system. The collision avoidance and formation maintenance control of the swarm system is realized by setting the collision avoidance and formation maintenance potential functions. Based on the proposed improved artificial potential field method, a new distributed controller for swarm system is designed and its stability is analyzed. Numerical simulations are conducted to prove the effectiveness of the proposed strategy.

IMPROVING BYZANTINE FAULT TOLERANCE IN SWARM ROBOTICS COLLECTIVE DECISION-MAKING SCENARIO VIA A NEW BLOCKCHAIN CONSENSUS ALGORITHM

Swarm robotics applies concepts of swarm intelligence to robotics. Discrete consensus achievement is one of the major behaviors found in swarm robotics. Various algorithms have been developed for discrete consensus achievement. However, existing discrete consensus achievement algorithms are vulnerable to Byzantine robots. Blockchain has been successfully used to mitigate the negative effect of Byzantine robots. Nevertheless, since the blockchain solution uses the Proof-of-Work blockchain consensus algorithm, it is vulnerable to the 51% attack. Besides, the swarm also takes longer to achieve consensus. This research proposes a novel blockchain consensus algorithm called Proof-of-Identity—which uses a private-public key pair and a swarm controller—to create a dynamically permissioned blockchain that would negate the 51%-attack problem associated with the Proof-of-Work algorithm while also reducing the consensus time. This proposed solution was tested against the classical solution and the existing blockchain solution using the collective perception scenario. Test results show that the Proof-of-Identity algorithm prevents the 51%-attack problem while improving the consensus time in comparison to the existing blockchain solution without affecting the exit probability.

Spatial Patterning of Micromotor Aggregation and Flux.

Using a spatially varying light pattern with light activated semi-conductor based magnetic TiO2 micromotors, we study the difference in micromotor flux between illuminated and non-illuminated regions in the presence and absence of an applied magnetic field. We find that the magnetic field enhances the flux of the motors which we attribute to a straightening of the micromotor trajectories which decreases the time they spend in the illuminated region. We also demonstrate spatially patterned light-induced aggregation of the micromotors and study its time evolution at various micromotor concentrations. Although light induced aggregation has been observed previously, spatial patterning of aggregation demonstrates a further means of control which could be relevant to swarm control or self-assembly applications. Overall, these results draw attention to the effect of trajectory shape on the flux of active colloids as well as the concentration dependence of aggregation and its time dependence within a spatially patterned region, which is not only pertinent to self-assembly and swarm control, but also provides insight into the behavior of active matter systems with spatially varying activity levels.

Intelligent Planning for Large-Scale Multi-Robot Coordination

Robots will play a crucial role in the future and need to work as a team in increasingly more complex applications. Advances in robotics have laid the hardware foundations for building large-scale multi-robot systems. But how to coordinate robots intelligently is a difficult problem. We believe that graph-search-based planning can systematically exploit the combinatorial structure of multi-robot coordination problems and efficiently generate solutions with rigorous guarantees on correctness, completeness, and solution quality. We started with one problem that is central to many multi-robot applications. Multi-Agent Path Finding (MAPF) is an NP-hard problem of planning collision-free paths for a team of agents while minimizing their travel times. We addressed the MAPF problem from both (1) a theoretical perspective by developing efficient algorithms to solve large MAPF instances with completeness and optimality guarantees via a variety of AI and optimization technologies, such as constraint reasoning, heuristic search, stochastic local search, and machine learning, and (2) an applicational perspective by developing algorithmic techniques for integrating MAPF with task planning and execution for various multi-robot systems, such as mobile robot coordination, traffic management, drone swarm control, multi-arm assembly, and character control in video games. This paper is part of the AAAI-23 New Faculty Highlights.

Dedicated Adaptive Particle Swarm Optimization Algorithm for Digital Twin Based Control Optimization of the Plug-In Hybrid Vehicle

The digital twin (DT) is a promising technology that will provide a cyber-physical platform to enable rapid product development boosted by artificial intelligence and big data. This paper focuses on DT-based optimization of the energy management strategy (EMS) for a plug-in hybrid vehicle (PHEV). A dedicated adaptive particle swarm optimization (DAPSO) algorithm is developed to enhance the optimality and trustworthiness of the DT-based EMS optimization in a variety of driving conditions. By using the chassis dynamometer data, the DAPSO is developed by incorporating the widely-used PSO algorithm with an adaptive swarm control strategy that coordinates the exploration and exploitation of the individual swarming particles following the global information extracted from the swarm. With the cross-validation testing under three worldwide driving conditions, this study determines the unified settings of the DAPSO algorithm for the PHEV application. Experimental evaluations are conducted by monitoring the PHEV’s performance using different EMSs with control thresholds optimized by the DAPSO and conventional PSO algorithms (baseline). The results show that by introducing DAPSO for DT-based EMS optimization, up to 24% improvement in cost function value (weighted sum of the fuel consumption and SoC sustaining error) and more than 49.1% reduction in computing time can be achieved compared to the baseline methods.