Multi-robot Task Allocation Research Articles

Q-Learning Based Failure Detection and Self-Recovery Algorithm for Multi-Robot Domains

Task allocation is the essential part of multi-robot coordination researches and it plays a significant role to achieve desired system performance. Uncertainties in multi-robot systems’ working environment due to nature of them are the major hurdle for perfect coordination. When learning-based task allocation approaches are used, firstly robots learn about their working environment and then they benefit from their experiences in future task allocation process. These approaches provide useful solutions as long as environmental conditions remain unchanged. If permanent changes in environment characteristics or some failure in multi-robot system occur undesirably e.g. in disaster response which is a good example to represent such cases, the previously-learned information becomes invalid. At this point, the most important mission is to detect the failure and to recover the system initial learning state. For this purpose, Q-learning based failure detection and self-recovery algorithm is proposed in this study. According to this approach, multi-robot system checks whether these variations permanent, then recover the system to learning state if it is required. So, it provides dynamic task allocation procedure having great advantages against unforeseen situations. The experimental results verify that the proposed algorithm offer efficient solutions for multi-robot task allocation problem even in systemic failure cases.DOI: 10.5755/j01.eie.25.1.22728

Auctions for multi-robot task allocation in communication limited environments

We consider the problem of multi-robot task allocation using auctions, and study how lossy communication between the auctioneer and bidders affects solution quality. We demonstrate both analytically and experimentally that even though many auction algorithms have similar performance when communication is perfect, different auctions degrade in different ways as communication quality decreases from perfect to nonexistent. Thus, if a multi-robot system is expected to encounter lossy communication, then the auction algorithm that it uses for task allocation must be chosen carefully. We compare six auction algorithms including: standard implementations of the Sequential Auction, Parallel Auction, Combinatorial Auction; a generalization of the Prim Allocation Auction called G-Prim; and two multi-round variants of a Repeated Parallel Auction. Variants of these auctions are also considered in which award information from previous rounds is rebroadcast by the auctioneer during later rounds. We consider a variety of valuation functions used by the bidders, including: the total and maximum distance traveled (for distance based cost functions), the expected profit or cost to a robot (assuming robots’ task values are drawn from a random distribution). Different auctioneer objectives are also evaluated, and include: maximizing profit (max sum), minimizing cost (min sum), and minimizing the maximum distance traveled by any particular robot (min max). In addition to the cost value functions that are used, we are also interested in fleet performance statistics such as the expected robot utilization rate, and the expected number of items won by each robot. Experiments are performed both in simulation and on real AscTec Pelican quad-rotor aircraft. In simulation, each algorithm is considered across communication qualities ranging from perfect to nonexistent. For the case of the distance-based cost functions, the performance of the auctions is compared using two different communication models: (1) a Bernoulli model and (2) the Gilbert–Elliot model. The particular auction that performs the best changes based on the the reliability of the communication between the bidders and the auctioneer. We find that G-Prim and its repeated variant perform relatively well when communication is poor, and that re-sending winner data in later rounds is an easy way improve the performance of multi-round auctions, in general.

PSO-based Dynamic Distributed Algorithm for Automatic Task Clustering in a Robotic Swarm

The Multi-Robot Task Allocation (MRTA) problem has recently become a key research topic. Task allocation is the problem of mapping tasks to robots, such that the most appropriate robot is selected to perform the most fitting task, leading to all tasks being optimally accomplished. Expanding the number of tasks and robots may cause the collaboration among the robots to become tougher. Since this process requires high computational time, this paper describes a technique that reduces the size of the explored state space, by partitioning the tasks into clusters. In real-world problems, the absence of information regarding the number of clusters is ordinarily occurring. Hence, a dynamic clustering is auspicious for partitioning the tasks to an appropriate number of clusters. In this paper, we address the problem of MRTA by putting forward a new simple, automatic and efficient clustering algorithm of the robots’ tasks based on a dynamic distributed particle swarm optimization, namely, ACD2PSO. Our approach is made out of two stages: stage I groups the tasks into clusters using the dynamic distributed particle swarm optimization (D2PSO) algorithm and stage II allocates the robots to the clusters. The assignment of robots to the clusters is represented as multiple traveling salesman problems (MTSP). Computational experiments were carried out to prove the effectiveness of our approach in term of clustering time, cost, and the MRTA time, compared to the distributed particle swarm optimization (dPSO) and genetic algorithm (GA). Thanks to the D2PSO algorithm, stagnation and local optima issues are avoided by adding assorted variety to the population, without losing the fast convergence of PSO.

Multi-Robot Cooperative Task Allocation With Definite Path-Conflict-Free Handling

Modeling and solving multi-robot task allocation with definite path-conflict-free handling is an important research, especially in real working environments. Some of the research lines are unable to obtain definite path-conflict-free solutions for multi-robot task allocations, such as using the penalty-term method in the fitness function to restrict the survival probabilities of the solutions with path conflicts. In some cases, these solutions are only able to satisfy the objective of minimizing task time. We formulate this problem based on grid maps, while focusing on the frequently used cooperative task allocation. In our model, two subtasks of each cooperative task must be executed by two robots, simultaneously. We propose vitality-driven genetic task allocation algorithm (VGTA), which is able to simultaneously minimize task time and realize definite conflict-free path planning. VGTA consists of local operators, such as random mutations, greedy crossovers, and vitality selection. Meanwhile, VGTA includes schedule conflict and path conflict handling strategies. In path conflict handling strategy, we not only consider the common path conflicts in a grid cell, but also focus on the path conflicts between robots when exchanging positions in the adjacent grid cells. Besides, we construct our benchmarks based on real working environments, such as factory, powerhouse, and airport environments. Experimental results indicate that VGTA’s search capability and computation cost are satisfactory. Meanwhile, its solutions are able to be really executed.

Application of Q-learning based on adaptive greedy considering negative rewards in football match system

Aiming at the problem that the multi-robot task allocation method in soccer system is easy to fall into the problem of local optimal solution and real-time performance, a new multi-robot task allocation method is proposed. First of all, in order to improve the speed and efficiency of finding optimal actions and make better use of the disadvantages that traditional Q-learning can't often propagate negative values, we propose a new way to propagate negative values, that is, Q-learning methods based on negative rewards. Next, in order to adapt to the dynamic external environment, an adaptive ε greedy method of which the mode of operation is judged by the ε value is proposed. This method is based on the classical ε-greedy. In the process of solving problems, ε can be adaptively changed as needed for a better balance of exploration and exploitation in reinforcement learning. Finally, we apply this method to the robot's football game system. It has been experimentally proven that dangerous actions can be avoided effectively by the Q-learning method which can spread negative rewards. The adaptive ε-greedy strategy can be used to adapt to the external environment better and faster so as to improve the speed of convergence.

Application of Q-learning based on adaptive greedy considering negative rewards in football match system

Aiming at the problem that the multi-robot task allocation method in soccer system is easy to fall into the problem of local optimal solution and real-time performance, a new multi-robot task allocation method is proposed. First of all, in order to improve the speed and efficiency of finding optimal actions and make better use of the disadvantages that traditional Q-learning can't often propagate negative values, we propose a new way to propagate negative values, that is, Q-learning methods based on negative rewards. Next, in order to adapt to the dynamic external environment, an adaptive e greedy method of which the mode of operation is judged by the e value is proposed. This method is based on the classical e-greedy. In the process of solving problems, e can be adaptively changed as needed for a better balance of exploration and exploitation in reinforcement learning. Finally, we apply this method to the robot's football game system. It has been experimentally proven that dangerous actions can be avoided effectively by the Q-learning method which can spread negative rewards. The adaptive e-greedy strategy can be used to adapt to the external environment better and faster so as to improve the speed of convergence.

Iterated Local Search for Time-extended Multi-robot Task Allocation with Spatio-temporal and Capacity Constraints

Abstract We propose a method for task allocation to multiple physical agents that works when tasks have temporal and spatial constraints and agents have different capacities. Assuming that the problem is over-constrained, we need to find allocations that maximize the number of tasks that can be done without violating any of the constraints. The contribution of this work is the study of a new multi-robot task allocation problem and the design and the experimental evaluation of our approach, an iterated local search that is suitable for time critical applications. We created test instances on which we experimentally show that our approach outperforms a state-of-the-art approach to a related problem. Our approach improves the baseline’s score on average by 2.35% and up to 10.53%, while responding in times shorter than the baseline’s, on average, 1.6 s and up to 5.5 s shorter. Furthermore, our approach is robust to run replication and is not very sensitive to parameters tuning.

Competency adjustment and workload balancing framework in multirobot task allocation

This article proposes a framework for competency adjustment and workload balancing in multirobot task allocation. Competency represents the ability of a robot to execute a task in terms of quality and cost, and workload balancing denotes distribution of workload among robots. This framework considers the quality and cost of a robot for a task and adjusts them in accordance with environmental changes. For workload balancing, the framework utilizes the concept of subsidy to encourage participation from the less active members of the robot team. The proposed framework is implemented in a simulated cleaning mission. Simulation results demonstrate that this framework can adjust the competency in accordance with environmental changes and distribute workload among robots in a balanced manner.

Multi-UUV Task Allocation with Limited Communication Range Based on Hoplites Framework

Multiple Unmanned Underwater Vehicles(UUVs) can be widely used in marine science expedition, ocean development and ocean engineering. And in the Unmanned Underwater Vehicles cluster, the Multi-Robot Task Allocation with communication range limitations is an important issue. In the past, many works were researched based on perfectly good communication conditions. However, in actual situations, especially underwater environments, communication range between robots is often limited. So in this paper, in order to ensure the communication range between the UUVs as much as possible, based on the Hoplites framework, we first design the communication keep revenue so that the UUVs not only perform the task but also try to ensure the communication range between the UUVs. Second, when there are individual tasks that are particularly far away from all other tasks, a communication relay node needs to be added between the UUV and other UUVs. We regard the communication relay node as a task, which is realized by using the active coordination mechanism of the Hoplites framework. Through the simulation environment of multiple UUVs, we conduct experiments and analysis of the algorithm, and the results show tangible performance gain of our methods over previous methods.

Label‐QoS switching protocol for quality of service assurance in dynamic swarm robot local network

SummaryThis paper presents the modeling and analysis of a proposed label–quality of service (QoS) switching protocol for heterogeneous robot in a swarm. Establishing the data communication and network in multirobot task allocation is an important aspect in robot collaboration. Instead of passing the data through the cloud network, each robot can be configured as a node in its swarm for intercommunication. This research was conducted to model and propose aggregation and classification methods in a swarm robot network inspired from multiprotocol label switching, namely, label‐QoS switching protocol. Each packet of data is forwarded with a proposed 2‐side label values that are concerned with addressing and QoS. This proposed protocol was applied in each of the node's routing, and it was set up with forwarding information table. The simulation and analysis were conducted in 2 situations: (1) with a constant n–label switch mobile robot and a number of packet data increasing with time and (2) with a constant number of packet and varying n–label switch mobile robot by time. With reference to the network parameter performances, it shows that the anomaly treatment by the proposed protocol is able to prioritize the data forwarding between the robots at the edge of the swarm with class of service although the robot community at the center is increasing or getting crowded.

Multi-objective Optimisation of Multi-robot Task Allocation with Precedence Constraints

Efficacy of the multi-robot systems depends on proper sequencing and optimal allocation of robots to the tasks. Focuses on deciding the optimal allocation of set-of-robots to a set-of-tasks with precedence constraints considering multiple objectives. Taguchi’s design of experiments based parameter tuned genetic algorithm (GA) is developed for generalised task allocation of single-task robots to multi-robot tasks. The developed methodology is tested for 16 scenarios by varying the number of robots and number of tasks. The scenarios were tested in a simulated environment with a maximum of 20 robots and 40 multi-robot foraging tasks. The tradeoff between performance measures for the allocations obtained through GA for different task levels was used to decide the optimal number of robots. It is evident that the tradeoffs occur at 20 per cent of performance measures and the optimal number of robot varies between 10 and 15 for almost all the task levels. This method shows good convergence and found that the precedence constraints affect the optimal number of robots required for a particular task level.

Route Planning Integrated Multi Objective Task Allocation for Reconfigurable Robot Teams Using Genetic Algorithm

This research work aims at multi objective optimization of integrated route planning and multi-robot task allocation for reconfigurable robot teams. Genetic Algorithm based methodology is used to minimize the overall task completion time for all the multi-robot tasks and to minimize the cumulative running time of all the robots. A modified matrix based chromosome is used to accommodate the robot information and task information for route planning integrated task allocation. The experimental validation is done with 3 robots and 4 tasks. For larger number of robots and tasks were simulated to perform route planning for maximum of 20 robots that would attend the maximum of 40 different multi-robot tasks. The results shows that the average task completion time per robot and average travel time per robot, decreases exponentially with increase in number of robots for fixed number of tasks. This method finds its application in allocating a robot teams to tasks and finding the best sequence for robots that work in coordination for material handling in hospital management, warehouse operations, military operations, cleaning tasks etc.

Hybrid Optimization-Based Approach for Multiple Intelligent Vehicles Requests Allocation

Self-driving cars are attracting significant attention during the last few years, which makes the technology advances jump fast and reach a point of having a number of automated vehicles on the roads. Therefore, the necessity of cooperative driving for these automated vehicles is exponentially increasing. One of the main issues in the cooperative driving world is the Multirobot Task Allocation (MRTA) problem. This paper addresses the MRTA problem, specifically for the problem of vehicles and requests allocation. The objective is to introduce a hybrid optimization-based approach to solve the problem of multiple intelligent vehicles requests allocation as an instance of MRTA problem, to find not only a feasible solution, but also an optimized one as per the objective function. Several test scenarios were implemented in order to evaluate the efficiency of the proposed approach. These scenarios are based on well-known benchmarks; thus a comparative study is conducted between the obtained results and the suboptimal results. The analysis of the experimental results shows that the proposed approach was successful in handling various scenarios, especially with the increasing number of vehicles and requests, which displays the proposed approach efficiency and performance.

A Comparative Study of Game-theoretical and Markov-chain-based Approaches to Division of Labour in a Robotic Swarm

This paper compares two swarm intelligence frameworks that we previously proposed for multi-robot task allocation problems: the game-theoretical approach based on anonymous hedonic games, called GRAPE, and the Markov-Chain-based approach under local information consistency assumption, called LICA-MC. We implement both frameworks into swarm distribution guidance problem, the objective of which is to distribute a swarm of robots into a set of tasks in proportion to the tasks’ demands, and then we perform extensive numerical experiments with various environmental settings. The statistical results show that LICA-MC provides excellent scalability regardless of the number of robots, whereas GRAPE is more efficient in terms of convergence time (especially when accommodating a moderate number of robots) as well as total travelling costs. Furthermore, this study investigates other implicit advantages of the frameworks such as mission suitability, additionally-built-in decision-making functions, and sensitivity to traffic congestion or robots’ mobility.

Robots in Retirement Homes: Person Search and Task Planning for a Group of Residents by a Team of Assistive Robots

The authors present a general multirobot task planning and execution architecture for a team of heterogeneous mobile robots that interact with multiple human users. The designed architecture is implemented in an environment where such robots provide daily assistance to residents in a retirement home setting. The robots are able to allocate and schedule activities throughout the day and find the appropriate residents with whom to engage in assistive activities. Upon finding users, the robot queries each user for his or her availability and interest in various activities. The authors then use constraint programming within a multirobot task allocation and scheduling system to plan and schedule the robot team to facilitate individual and group activities, ensuring consistency with user-expressed availability and activity preferences. The authors test the components of the architecture on a physical multirobot system to verify the utility of the design. Experiments indicate the design can effectively plan and execute assistive activities for multiple users.

EXPERIENCED TASK-BASED MULTI ROBOT TASK ALLOCATION

In multi robot system applications, it is possible that the robots transform their past experiences into useful information which will be used for next task allocation processes by using learning-based task allocation mechanisms. The major disadvantages of multi-robot Q-learning algorithm are huge learning space and computational cost due to generalized state and joint action spaces of robots. In this study, experienced task-based multi robot task allocation approach is proposed. According to this approach, robots believe to be experienced about the tasks most frequently done. Robots prefer to do these tasks rather than the inexperienced ones. Then, robots refuse to execute inexperienced tasks over time. This means that the system has reduced learning space. The proposed approach plays a crucial role to achieve required system performance and provides effective solutions to learning space dimensions. The effectiveness of the proposed algorithm is demonstrated by simulations on multi-robot task allocation problem.

Toward a Possibilistic Swarm Multi-robot Task Allocation: Theoretical and Experimental Results

Selecting the best task to execute (task allocation problem) is one of the main problems in multi-robot systems. Typical ways to address this problem are based on swarm intelligence and very especially using the so-called response threshold method. In the aforementioned method a robot has a certain probability of executing a task which depends on a task threshold response and a task stimulus. Nevertheless, response threshold method cannot be extended in a natural way to allocate more than two tasks when the theoretical basis is provided by probability theory. Motivated by this fact, this paper leaves the probabilistic approach to the problem and provides a first theoretical framework towards a possibilistic approach. Thus, task allocation problem is addressed using fuzzy Markov chains instead of probabilistic processes. This paper demonstrates that fuzzy Markov chains associated to a task allocation problem can converge to a stationary stage in a finite number of steps. In contrast, the probabilistic processes only can converge asymptotically, i.e. the number of steps is not bounded in general. Moreover, fuzzy Markov chains predicts in a better way the future behavior of the system in the presence of vagueness when measuring distances. The simulations performed confirm the theoretical results and show how the number of steps needed to get a stable state with fuzzy Markov chains is reduced more than 10 times and the system's behavior prediction can be improved more than a 60% compared to probabilistic approaches.

Multirobot task allocation based on an improved particle swarm optimization approach

Due to its complexity and non-deterministic polynomial-time hard characteristic, multirobot task allocation problem remains a challenging issue in the field of cooperative robotics. Thanks to its easy implementation and promising convergence speed, the particle swarm optimization method has recently aroused increasing research interest in the area of multirobot task allocation problem. However, the efficiency of the standard particle swarm optimization is hindered by several deficiencies such as the inefficient capabilities in balancing exploration and exploitation, as well as the high likelihood of plunging into stagnation. Aiming at enhancing the performance of particle swarm optimization via remedying these two drawbacks, this paper proposes an improved particle swarm optimization method, which integrates standard particle swarm optimization 2011 with evolutionary game theory. To prevent particles being locked into stagnation, particles in the proposed particle swarm optimization first adopt the updating rules of standard particle swarm optimization 2011 to undertake their movements. Subsequently, attempting to well trade off the exploration and exploitation capabilities of particles, a novel self-adaptive strategy, which is determined by the evolutionary stable strategies of evolutionary game theory and the iteration number of particle swarm optimization, is presented to adaptively adjust the main control parameters of particles in the proposed particle swarm optimization. Since the convergence of particle swarm optimization remains paramount and dramatically affects the performance of particle swarm optimization, this paper also analytically investigates the convergence of the proposed particle swarm optimization and provides a convergence-guaranteed parameter selection principle for the proposed method. Finally, leveraging the development of the proposed particle swarm optimization, this paper completes the design of a new particle swarm optimization–based multirobot task allocation method. The performance of the new particle swarm optimization–based multirobot task allocation method is tested through three different allocation cases against four well-known evolutionary methods. Experimental results confirm that the proposed method generally outperforms its contenders in terms of the solution quality. Moreover, the proposed method performs slightly better than the majority of its peers as far as the computation time is concerned.

DTTA - Distributed, Time-division Multiple Access based Task Allocation Framework for Swarm Robots

Swarm robotic systems, unlike traditional multi-robotic systems, deploy number of cost effective robots which can co-operate, aggregate to form patterns/formations and accomplish missions beyond the capabilities of individual robot. In the event of fire, mine collapse or disasters like earthquake, swarm of robots can enter the area, conduct rescue operations, collect images and convey locations of interest to the rescue team and enable them to plan their approach in advance. Task allocation among members of the swarm is a critical and challenging problem to be addressed. DTTA- a distributed, Time-division multiple access (TDMA) based task allocation framework is proposed for swarm of robots which can be utilised to solve any of the 8 different types of task allocation problem identified by Gerkey and Mataric´. DTTA is reactive and supports task migration via extended task assignments to complete the mission in case of failure of the assigned robot to complete the task. DTTA can be utilised for any kind of robot in land or for co-operative systems comprising of land robots and air-borne drones. Dependencies with other layers of the protocol stack were identified and a quantitative analysis of communication and computational complexity is provided. To our knowledge this is the first work to be reported on task allocation for clustered scalable networks suitable for handling all 8 types of multi-robot task allocation problem. Effectiveness and feasibility of deploying DTTA in real world scenarios is demonstrated by testing the framework for two diverse application scenarios.

An Evolutionary Algorithm Based Framework for Task Allocation in Multi-Robot Teams

Multi-Robot Task Allocation (MRTA) has no formal framework which could provide solutions covering different domains within the MRTA taxonomy without changing the optimization scheme. This research aims to develop a novel framework using evolutionary computing. The study proposes a modular approach towards developing this framework in which individual problem types of the MRTA taxonomy are solved one at a time. The performance of the framework will be evaluated against the popular approaches suggested for each problem type.