Goal Position Research Articles

The Task Decomposition and Dedicated Reward-System-Based Reinforcement Learning Algorithm for Pick-and-Place

This paper proposes a task decomposition and dedicated reward-system-based reinforcement learning algorithm for the Pick-and-Place task, which is one of the high-level tasks of robot manipulators. The proposed method decomposes the Pick-and-Place task into three subtasks: two reaching tasks and one grasping task. One of the two reaching tasks is approaching the object, and the other is reaching the place position. These two reaching tasks are carried out using each optimal policy of the agents which are trained using Soft Actor-Critic (SAC). Different from the two reaching tasks, the grasping is implemented via simple logic which is easily designable but may result in improper gripping. To assist the grasping task properly, a dedicated reward system for approaching the object is designed through using individual axis-based weights. To verify the validity of the proposed method, wecarry out various experiments in the MuJoCo physics engine with the Robosuite framework. According to the simulation results of four trials, the robot manipulator picked up and released the object in the goal position with an average success rate of 93.2%.

Multirobot Navigation Using Partially Observable Markov Decision Processes with Belief-Based Rewards

For the safe deployment of multirobot systems, accurate position estimation is necessary. We present a multirobot navigation algorithm that accomplishes the dual objective of reaching designated goal positions and maintaining low position uncertainty. We pose the problem as a -partially observable Markov decision process (POMDP) to capture position uncertainty through a belief-based reward. The multirobot system employs a centralized extended Kalman filter (EKF) for state estimation, whose output is used as the observation as well as the belief in the POMDP. Using the EKF output as the observation and belief allows for the integration of the EKF in the action planning module of the system. To solve the POMDP, we adapt the online information particle filter tree algorithm to be compatible with the EKF closed-form output being used as both the observation and belief. We present results for the proposed navigation algorithm that show emergent behaviors for a multirobot system, where the robots move to provide inter-ranging measurements that minimize position uncertainty for all robots in the system. We also demonstrate that the proposed method outperforms a state-of-the-art solver (partially observable Monte Carlo planning with observation widening) and a direct-to-goal planner in terms of minimizing position uncertainty.

Leader-Based Flocking of Multiple Swarm Robots in Underwater Environments.

Considering underwater environments, this paper tackles flocking of multiple swarm robots utilizing one leader. The mission of swarm robots is to reach their goal while not colliding with a priori unknown 3D obstacles. In addition, the communication link among the robots needs to be preserved during the maneuver. Only the leader has sensors for localizing itself while accessing the global goal position. Every robot, except for the leader, can measure the relative position and the ID of its neighboring robots by utilizing proximity sensors such as Ultra-Short BaseLine acoustic positioning (USBL) sensors. Under the proposed flocking controls, multiple robots flock inside a 3D virtual sphere while preserving communication connectivity with the leader. If necessary, all robots rendezvous at the leader to increase connectivity among the robots. The leader herds all robots to reach the goal safely, while the network connectivity is maintained in cluttered underwater environments. To the best of our knowledge, our article is novel in developing underwater flocking controls utilizing one leader, so that a swarm of robots can safely flock to the goal in a priori unknown cluttered environments. MATLAB simulations were utilized to validate the proposed flocking controls in underwater environments with many obstacles.

Long-Term Dynamic Window Approach for Kinodynamic Local Planning in Static and Crowd Environments

Local planning for a differential wheeled robot is designed to generate kinodynamic feasible actions that guide the robot to a goal position along the navigation path while avoiding obstacles. Reactive, predictive, and learning-based methods are widely used in local planning. However, few of them can fit static and crowd environments while satisfying kinodynamic constraints simultaneously. To solve this problem, we propose a novel local planning method. The method applies a long-term dynamic window approach to generate an initial trajectory and then optimizes it with graph optimization. The method can plan actions under the robot's kinodynamic constraints in real time while allowing the generated actions to be safer and more jitterless. Experimental results show that the proposed method adapts well to crowd and static environments and outperforms most state-of-the-art approaches.

A polynomial-time hybrid solver for multi-agent motion navigation against deadlocks

In the cooperative multi-agent pathfinding and motion planning, given a unique start position and a unique goal position for each agent, all agents are able to pursue their own goals without colliding with each other. To aim at realizing the collision-free motion of the agents within the tractable time, this work proposes a polynomial-time solver, called the HBD-AOI, hybridizing centralized and decentralized schemes. Firstly, an algorithm of centralized pathfinding is utilized to plan the optimal paths of all agents. Afterwards, each of the agents updates the local motion pattern to tracks its own planned waypoints with the obstacle avoidance in a decentralized manner. Furthermore, to resolve unavoidable egoistic conflicts occurring in the decentralized scheme, a centralized intervener with the route replanning is invoked to coach the involved agents to abort the existing deadlocks. Bounded by an amount of time, the performances of the proposed and benchmarked algorithms are simulated on the same instance, from the evaluated testbeds that consists of various maps and scenarios. In the simulations, it is proved that this work outperforms other benchmarked algorithms for all presented instances in the term of the success rate. The experimental results are also demonstrated to verify the feasibility of the proposed methodology.

A decoupling method for solving the multi-agent path finding problem

Many existing multi-agent path finding (MAPF) solvers focus on completeness, speed, or optimization. However, completeness and rapidity are usually in conflict with each other, which makes these algorithms far from satisfactory in practical applications. Motivated by this realistic requirement, we propose an efficient decoupling method to accelerate the solution of large MAPF problems. First, we define the concept of ‘non-essential vertex’-vertices which are not needed to solve a MAPF problem, and a scheme to identify them. Then, a decoupling scheme based on ‘non-essential vertex’ is proposed, which will assign higher priorities to agents whose goal positions are non-essential vertices and lower priorities to agents whose start positions are non-essential vertices. That is, invoking our decoupling algorithm can decouple any given MAPF problem into three subproblems while maintaining the completeness of the solution. All three sub-MAPF problems can be solved sequentially by a complete solver (e.g., CBS or EECBS, etc.), and two of them can also be solved by a prioritized planning algorithm. We have conducted several experiments in different workspaces, and the statistical results show that the proposed decoupling method significantly improves the speed and success rate of existing MAPF solvers with almost no degradation in solution quality when solving problems with high agent density. In addition, the solving efficiency can be further improved if the prioritized planning algorithm is invoked to solve the first and third sub-MAPF problems.

Cable failure tolerant control and planning in a planar reconfigurable cable driven parallel robot.

The addition of geometric reconfigurability in a cable driven parallel robot (CDPR) introduces kinematic redundancies which can be exploited for manipulating structural and mechanical properties of the robot through redundancy resolution. In the event of a cable failure, a reconfigurable CDPR (rCDPR) can also realign its geometric arrangement to overcome the effects of cable failure and recover the original expected trajectory and complete the trajectory tracking task. In this paper we discuss a fault tolerant control (FTC) framework that relies on an Interactive Multiple Model (IMM) adaptive estimation filter for simultaneous fault detection and diagnosis (FDD) and task recovery. The redundancy resolution scheme for the kinematically redundant CDPR takes into account singularity avoidance, manipulability and wrench quality maximization during trajectory tracking. We further introduce a trajectory tracking methodology that enables the automatic task recovery algorithm to consistently return to the point of failure. This is particularly useful for applications where the planned trajectory is of greater importance than the goal positions, such as painting, welding or 3D printing applications. The proposed control framework is validated in simulation on a planar rCDPR with elastic cables and parameter uncertainties to introduce modeled and unmodeled dynamics in the system as it tracks a complete trajectory despite the occurrence of multiple cable failures. As cables fail one by one, the robot topology changes from an over-constrained to a fully constrained and then an under-constrained CDPR. The framework is applied with a constant-velocity kinematic feedforward controller which has the advantage of generating steady-state inputs despite dynamic oscillations during cable failures, as well as a Linear Quadratic Regulator (LQR) feedback controller to locally dampen these oscillations.

A Multigoal Path-Planning Approach for Explosive Ordnance Disposal Robots Based on Bidirectional Dynamic Weighted-A* and Learn Memory-Swap Sequence PSO Algorithm

In order to protect people’s lives and property, increasing numbers of explosive disposal robots have been developed. It is necessary for an explosive ordinance disposal (EOD) robot to quickly detect all explosives, especially when the location of the explosives is unknown. To achieve this goal, we propose a bidirectional dynamic weighted-A star (BD-A*) algorithm and a learn memory-swap sequence particle swarm optimization (LM-SSPSO) algorithm. Firstly, in the BD-A* algorithm, our aim is to obtain the shortest distance path between any two goal positions, considering computation time optimization. We optimize the computation time by introducing a bidirectional search and a dynamic OpenList cost weight strategy. Secondly, the search-adjacent nodes are extended to obtain a shorter path. Thirdly, by using the LM-SSPSO algorithm, we aim to plan the shortest distance path that traverses all goal positions. The problem is similar to the symmetric traveling salesman problem (TSP). We introduce the swap sequence strategy into the traditional PSO and optimize the whole PSO process by imitating human learning and memory behaviors. Fourthly, to verify the performance of the proposed algorithm, we begin by comparing the improved A* with traditional A* over different resolutions, weight coefficients, and nodes. The hybrid PSO algorithm is also compared with other intelligent algorithms. Finally, different environment maps are also discussed to further verify the performance of the algorithm. The simulation results demonstrate that our improved A* algorithm has superior performance by finding the shortest distance with less computational time. In the simulation results for LM-SSPSO, the convergence rate significantly improves, and the improved algorithm is more likely to obtain the optimal path.

Robot Navigation in Complex Workspaces Employing Harmonic Maps and Adaptive Artificial Potential Fields.

In this work, we address the single robot navigation problem within a planar and arbitrarily connected workspace. In particular, we present an algorithm that transforms any static, compact, planar workspace of arbitrary connectedness and shape to a disk, where the navigation problem can be easily solved. Our solution benefits from the fact that it only requires a fine representation of the workspace boundary (i.e., a set of points), which is easily obtained in practice via SLAM. The proposed transformation, combined with a workspace decomposition strategy that reduces the computational complexity, has been exhaustively tested and has shown excellent performance in complex workspaces. A motion control scheme is also provided for the class of non-holonomic robots with unicycle kinematics, which are commonly used in most industrial applications. Moreover, the tuning of the underlying control parameters is rather straightforward as it affects only the shape of the resulted trajectories and not the critical specifications of collision avoidance and convergence to the goal position. Finally, we validate the efficacy of the proposed navigation strategy via extensive simulations and experimental studies.

Motion duration is overestimated behind an occluder in action and perception tasks.

Motion estimation behind an occluder is a common task in situations like crossing the street or passing another car. People tend to overestimate the duration of an object's motion when it gets occluded for subsecond motion durations. Here, we explored (a) whether this bias depended on the type of interceptive action: discrete keypress versus continuous reach and (b) whether it was present in a perception task without an interceptive action. We used a prediction-motion task and presented a bar moving across the screen with a constant velocity that later became occluded. In the action task, participants stopped the occluded bar when they thought the bar reached the goal position via keypress or reach. They were more likely to stop the bar after it passed the goal position regardless of the action type, suggesting that the duration of occluded motion was overestimated (or its speed was underestimated). In the perception task, where participants judged whether a tone was presented before or after the bar reached the goal position, a similar bias was observed. In both tasks, the bias was near constant across motion durations and directions and grew over trials. We speculate that this robust bias may be due to a temporal illusion, Bayesian slow-motion prior, or the processing of the visible-occluded boundary crossing. Understanding its exact mechanism, the conditions on which it depends, and the relative roles of speed and time perception requires further research.

Animation generation for object transportation with a rope using deep reinforcement learning

AbstractThis article presents a reinforcement learning‐based approach to generate animation in which two agents use a rope to collaboratively transport a block. The challenge is that the agents need to master several skills, including approaching the block, using the rope to wrap around it, and then moving the block to a predefined goal position. We propose several reward terms to learn the transportation policy and the adjustment policy that govern the skills of the agents. Experiment results showed that the proposed approach was able to generate various animations in different settings, including rope lengths, block sizes, and block shapes. An ablation test revealed the effects of the reward terms. We also investigated factors that affected the performance of the two policies.

Multiagent Manuvering with the Use of Reinforcement Learning

This paper presents an approach for defining, solving, and implementing dynamic cooperative maneuver problems in autonomous driving applications. The formulation of these problems considers a set of cooperating cars as part of a multiagent system. A reinforcement learning technique is applied to find a suboptimal policy. The key role in the presented approach is a multiagent maneuvering environment that allows for the simulation of car-like agents within an obstacle-constrained space. Each of the agents is tasked with reaching an individual goal, defined as a specific location in space. The policy is determined during the reinforcement learning process to reach a predetermined goal position for each of the simulated cars. In the experiments, three road scenarios—zipper, bottleneck, and crossroads—were used. The trained policy has been successful in solving the cooperation problem in all scenarios and the positive effects of applying shared rewards between agents have been presented and studied. The results obtained in this work provide a window of opportunity for various automotive applications.

Development and evaluation of a space robot prototype equipped with a cable-driven manipulator

This research focuses on developing and evaluating a free-flying space robot equipped with a cable-driven manipulator and an adaptive gripper. First, a transmission joint comprising a pair of gears and pulleys is designed and actuated by motors located in the first link of the manipulator. Afterward, based on the kinematics and dynamics models of the robot, a mission distribution control scheme is proposed for the robot to drive its end-effector to achieve the desired position and attitude for capturing space targets. Furthermore, exploiting this strategy, considerable comparison simulations between two space robots equipped with a traditional manipulator and a cable-driven arm are conducted. Wherein, the goal position and attitude are selected through a uniform sampling method. For the same capture mission, energy consumptions of the base and manipulator as well as state errors of the base are taken as cost functions for performance evaluation. Finally, a planar target capture case study using the adaptive gripper and a spatial motion test with the help of an air-bearing simulator are carried out to demonstrate the efficiency of the presented cable-driven space robot.

Artificial Potential Field Path Planning Algorithm in Differential Drive Mobile Robot Platform for Dynamic Environment

Mobile robots need path-planning abilities to achieve a collision-free trajectory. Obstacles between the robot and the goal position must be passed without crashing into them. The Artificial Potential Field (APF) algorithm is a method for robot path planning that is usually used to control the robot for avoiding obstacles in front of the robot. The APF algorithm consists of an attractive potential field and a repulsive potential field. The attractive potential fields work based on the predetermined goals that are generated to attract the robot to achieve the goal position. Apart from it, the obstacle generates a repulsive potential field to push the robot away from the obstacle. The robot's localization in producing the robot's position is generated by the differential drive kinematic equations of the mobile robot based on encoder and gyroscope data. In addition, the mapping of the robot's work environment is embedded in the robot's memory. According to the experiment's results, the mobile robot's differential drive can pass through existing obstacles. In this research, four test environments represent different obstacles in each environment. The track length is 1.5 meters. The robot's tolerance to the goal is 0.1 m, so when the robot is in the 1.41 m position, the robot's speed is 0 rpm. The safe distance between the robot and the obstacle is 0.2 m, so the robot will find a route to get away from the obstacle when the robot reaches that safe distance. The speed of the resulting robot decreases as the distance between the robot and the destination gets closer according to the differential drive kinematics equation of the mobile robot.

Swarm Cooperative Navigation Using Centralized Training and Decentralized Execution

The demand for autonomous UAV swarm operations has been on the rise following the success of UAVs in various challenging tasks. Yet conventional swarm control approaches are inadequate for coping with swarm scalability, computational requirements, and real-time performance. In this paper, we demonstrate the capability of emerging multi-agent reinforcement learning (MARL) approaches to successfully and efficiently make sequential decisions during UAV swarm collaborative tasks. We propose a scalable, real-time, MARL approach for UAV collaborative navigation where members of the swarm have to arrive at target locations at the same time. Centralized training and decentralized execution (CTDE) are used to achieve this, where a combination of negative and positive reinforcement is employed in the reward function. Curriculum learning is used to facilitate the sought performance, especially due to the high complexity of the problem which requires extensive exploration. A UAV model that highly resembles the respective physical platform is used for training the proposed framework to make training and testing realistic. The scalability of the platform to various swarm sizes, speeds, goal positions, environment dimensions, and UAV masses has been showcased in (1) a load drop-off scenario, and (2) UAV swarm formation without requiring any re-training or fine-tuning of the agents. The obtained simulation results have proven the effectiveness and generalizability of our proposed MARL framework for cooperative UAV navigation.

Robot Programming from a Single Demonstration for High Precision Industrial Insertion

We propose a novel approach for robotic industrial insertion tasks using the Programming by Demonstration technique. Our method allows robots to learn a high-precision task by observing human demonstration once, without requiring any prior knowledge of the object. We introduce an Imitated-to-Finetuned approach that generates imitated approach trajectories by cloning the human hand's movements and then fine-tunes the goal position with a visual servoing approach. To identify features on the object used in visual servoing, we model object tracking as the moving object detection problem, separating each demonstration video frame into the moving foreground that includes the object and demonstrator's hand and the static background. Then a hand keypoints estimation function is used to remove the redundant features on the hand. The experiment shows that the proposed method can make robots learn precision industrial insertion tasks from a single human demonstration.

Conflict-Based Search for Optimal Multi-Agent Pathfinding

We are assumed a customary of mediators in the multi-agent pathfinding problem (MAPF), each of which has its own start and goal positions. The objective is to discovery paths for each agent without causing any collisions. The majority of previous work on the optimal solution to this issue has useful single-agent exploration variations of the A* algorithm and treated each agent as a single "joint agent." The Conflict Based Search (CBS) optimum multi-agent pathfinding algorithm is the subject of this paper. CBS has two layer algorithm that does not use a single "joint agent" model to solve the problem. A Conflict Tree (CT), which is a tree based on conflicts between individual agents, is the high-level search engine. In the CT, each node represents a set of constraints on the agents' motion. To meet the high-level CT node's constraints, quick single-agent searches are carried out at the low level. CBS can frequently examine fewer states than A* while still maintaining optimality thanks to this two-level formulation. We discuss CBS's advantages and disadvantages. The Meta-Agent CBS (MA-CBS) algorithm is also discussed.MA-CBS is a broadening of CBS.MA-CBS, in contrast to basic CBS, permits low-level single-agent searches. Instead, agents can be combined into small groups of joint agents with MA-CBS. Performance is further enhanced and some of the drawbacks of basic CBS are reduced as a result. In point of fact, MA-CBS is a framework that can be layered on top of any MAPF solver that is both complete and optimal to boost its performance. A speedup of up to an order of magnitude over previous methods is demonstrated by experimental results on various problems. We are given a set of agents in the multi-agent pathfinding problem (MAPF), each of which has its own start and goal positions. The objective is to find paths for each agent without causing any collisions. The majority of previous work on the optimal solution to this issue has applied single-agent search variants of the A* algorithm and treated each agent as a single "joint agent." The Conflict Based Search (CBS) optimal multi-agent pathfinding algorithm is the subject of this paper. CBS is a two-level algorithm that does not use a single "joint agent" model to solve the problem. A Conflict Tree (CT), which is a tree based on conflicts between individual agents, is the high-level search engine. In the CT, each node represents a set of constraints on the agents' motion. To meet the high-level CT node's constraints, quick single-agent searches are carried out at the low level. CBS can frequently examine fewer states than A* while still maintaining optimality thanks to this two-level formulation. We discuss CBS's advantages and disadvantages. The Meta-Agent CBS (MA-CBS) algorithm is also discussed.MA-CBS is a broadening of CBS.MA-CBS, in contrast to basic CBS, permits low-level single-agent searches. Instead, agents can be combined into small groups of joint agents with MA-CBS. Performance is further enhanced and some of the drawbacks of basic CBS are reduced as a result. In point of fact, MA-CBS is a framework that can be layered on top of any MAPF solver that is both complete and optimal to boost its performance. A speedup of up to an order of magnitude over previous methods is demonstrated by experimental results on various problems.

A modified cuckoo search algorithm implemented with SCA and PSO for multi-robot cooperation and path planning

The paper puts forward an intelligent approach that deals with the computation of an optimal path with collision avoidance for the stick-carrying twin moving from a pre-assumed start position to a predefined goal position. It has been solved through the efficient implementation of modified cuckoo search, sine cosine algorithm, and particle swarm optimization to design a hybrid algorithm aimed at using the communal advantages of the search and position update ability of these algorithms. The benefits are realized by integrating the egg-laying behavior of the cuckoo species to achieve an efficient global search strategy with modified parameters, local search strategy of particle swarm optimization, and greedy approach of sine cosine algorithm. The proposed algorithm is validated using 10 standard benchmark functions, computer simulation using C language, and real robot platform using Epuck robot to illustrate minimal time, shortest distance, collision avoidance, path smoothness, synchronized action, and reduced energy usage in terms of the path traveled, execution time, the number of steps, and the number of turns in the static as well as the dynamic environment.

An Energy Efficient Jumping Drone - A Simple Projectile Motion Approach

Jumping robots are interesting devices that offer several advantages in terms of navigation about cluttered environments. However, they present unique challenges with respect to their design and control. In this paper, we propose a design that uses four planar thrusters/propellers on a jumping robot. Such a system provides better maneuverability due to agility provided by the propellers to guide the motion. From energy consumption perspective, we use the gravity for free fall and a spring mechanism to execute the jumping motion without loss of much energy on impact. The primary contribution of this paper is developing a novel navigation and control method for such jumping robots to go to the desired goal. In this paper, we present a projectile motion planning approach for the control of the proposed system. We propose proportional (P) and proportional-derivative (PD) controllers that compute the launch velocity required for the jumping drone after impact with the ground to follow a projectile motion in each jump to reach the goal position. The jumping drone bounces after impact with the ground, and the drone is then actuated for a certain time till it attains the required launch velocity after which it is made to move freely under the influence of gravity. Such a system shows significant reduction of energy consumption (by 81.35%) as compared to a normal drone navigating to the same goal location. Simulation results validate the effectiveness of the proposed approach.

Navigation Among Movable Obstacles via Multi-Object Pushing Into Storage Zones

With the majority of mobile robot path planning methods being focused on obstacle avoidance, this paper, studies the problem of Navigation Among Movable Obstacles (NAMO) in an unknown environment, with static (i.e., that cannot be moved by a robot) and movable (i.e., that can be moved by a robot) objects. In particular, we focus on a specific instance of the NAMO problem in which the obstacles have to be moved to predefined storage zones. To tackle this problem, we propose an online planning algorithm that allows the robot to reach the desired goal position while detecting movable objects with the objective to push them towards storage zones to shorten the planned path. Moreover, we tackle the challenging problem where an obstacle might block the movability of another one, and thus, a combined displacement plan needs to be applied. To demonstrate the new algorithm’s correctness and efficiency, we report experimental results on various challenging path planning scenarios. The presented method has significantly better time performance than the baseline, while also introducing multiple novel functionalities for the NAMO problem.